-- Logs begin at Tue 2022-08-09 17:36:36 UTC, end at Tue 2022-08-09 19:11:41 UTC. -- Aug 09 18:28:40.619204 dv-cinder-826608-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 09 18:28:40.626443 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 09 18:28:40.628453 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 9 18:28:40 2022] *** Aug 09 18:28:40.628453 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 09 18:28:40.628453 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 09 18:28:40.628453 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: nodename: dv-cinder-826608-9-iscsi Aug 09 18:28:40.628453 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: machine: x86_64 Aug 09 18:28:40.628453 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: clock source: unix Aug 09 18:28:40.628453 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: pcre jit disabled Aug 09 18:28:40.628453 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: detected number of CPU cores: 4 Aug 09 18:28:40.628453 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: current working directory: / Aug 09 18:28:40.628453 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: detected binary path: /usr/bin/uwsgi-core Aug 09 18:28:40.628453 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: your processes number limit is 47680 Aug 09 18:28:40.629230 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: your memory page size is 4096 bytes Aug 09 18:28:40.629230 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: detected max file descriptor number: 2048 Aug 09 18:28:40.629230 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: lock engine: pthread robust mutexes Aug 09 18:28:40.629230 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: thunder lock: enabled Aug 09 18:28:40.629230 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 09 18:28:40.629230 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 09 18:28:40.669067 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: Python main interpreter initialized at 0x56376efcf9c0 Aug 09 18:28:40.669067 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: python threads support enabled Aug 09 18:28:40.669067 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: your server socket listen backlog is limited to 100 connections Aug 09 18:28:40.669067 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: your mercy for graceful operations on workers is 90 seconds Aug 09 18:28:40.669435 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: mapped 403077 bytes (393 KB) for 2 cores Aug 09 18:28:40.669562 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: *** Operational MODE: preforking *** Aug 09 18:28:40.669835 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: *** uWSGI is running in multiple interpreter mode *** Aug 09 18:28:40.669835 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: spawned uWSGI master process (pid: 80431) Aug 09 18:28:40.670048 dv-cinder-826608-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 09 18:28:40.671829 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: spawned uWSGI worker 1 (pid: 80445, cores: 1) Aug 09 18:28:40.671829 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: spawned uWSGI worker 2 (pid: 80446, cores: 1) Aug 09 18:28:40.671829 dv-cinder-826608-9-iscsi devstack@keystone.service[80431]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 09 18:28:46.247055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 18:28:46.247055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: from cryptography.utils import int_from_bytes Aug 09 18:28:46.247875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 18:28:46.247875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: from cryptography.utils import int_from_bytes Aug 09 18:28:46.248329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 18:28:46.248329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: from cryptography.utils import int_from_bytes Aug 09 18:28:46.248464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 18:28:46.248464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: from cryptography.utils import int_from_bytes Aug 09 18:28:46.297682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 18:28:46.297878 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 18:28:46.298053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] command line args: [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 18:28:46.298326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 18:28:46.298505 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ================================================================================ {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 18:28:46.298671 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 18:28:46.298812 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] admin_token = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.298895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 18:28:46.298976 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] config_dir = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.299055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] command line args: [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 18:28:46.299168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.299320 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 18:28:46.299413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] config_source = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.299492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ================================================================================ {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 18:28:46.299569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] debug = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.299701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.299978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] admin_token = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.299978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] config_dir = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.300102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.300225 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.300301 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.300374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] config_source = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.300489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.300573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] debug = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.300734 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.300837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.300977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.301137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.301225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_config_append = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.301335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.301419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.301484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_dir = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.301637 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.301711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_file = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.301825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.301900 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.302064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.302179 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.302256 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_config_append = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.302374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.302472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.302577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_dir = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.302658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.302771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_file = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.302879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.302959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.303045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.303173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.303267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.303420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.303500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.303659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.303741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.303869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.303949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.304316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.304412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.304557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.304656 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.304806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.304900 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.305042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.305125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.305277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.305357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.305640 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.305833 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.305925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.306005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] publish_errors = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.306118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.306245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.306368 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.306480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.306656 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.306744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.306903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.307168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.307298 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.307387 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] publish_errors = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.307465 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] standard_threads = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.307575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.307670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.307809 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.307895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.307989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.308067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.308181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] use_journal = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.308266 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.308345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] use_json = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.308460 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.308545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] use_stderr = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.308677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] use_syslog = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.308755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] standard_threads = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.308909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.308988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.309152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.309229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.309330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.309415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.309584 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] use_journal = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.309667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.309739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] use_json = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.309862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] use_stderr = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.309948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.310026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] use_syslog = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.310133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.310260 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:28:46.310373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.310519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.310674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.310752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.310916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.310995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.311235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.311312 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.311487 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.311564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.311727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.311895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.311979 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.312202 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.312279 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.312452 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.312540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.312700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.312784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.312950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.313081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.313186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.313379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.313466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.313686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.313765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.313936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.314027 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.314185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.314295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.314424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.314619 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.314705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.314921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.314999 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.315168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.315269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.315406 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.315645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.315798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.315935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.316013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.316190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.316360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.316459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.316612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.316699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.316886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.316973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.317118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.317196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.317356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.317502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.317634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.317839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.317919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.318105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.318194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.318338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.318457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.318587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.318752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.318830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.319061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.319140 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.319311 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.319396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.319543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.319650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.319781 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.319888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.320011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.320122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.320250 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.320358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.320465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.320580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.320675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.320797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.320907 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.320993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.321138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.321220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.321370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.321453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.321626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.321726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.321866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.321947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.322097 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.322179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.322332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.322465 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.322586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.322792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.322869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.322940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.323068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.323141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.323265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.323353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.323495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.323580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.323731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.323812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.323967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.324059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.324203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.324326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.324433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.324510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.324664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.324747 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.324894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.324976 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.325133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.325221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.325366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.325445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.325627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.325723 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.325860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.325948 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.326102 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.326192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.326343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.326423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.326572 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.326649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.326809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.326900 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.327045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.327124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.327314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.327391 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.327551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.327704 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.327780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.327850 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.328058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.328137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.328299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.328387 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.328537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.328616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.328774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.328852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.329012 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.329102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.329248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.329327 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.329484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.329581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.329748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.329827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.329985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.330064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.330219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.330297 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.330458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.330538 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.330690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.330775 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.330925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.331003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.331182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.331261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.331420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.331500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.331661 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.331738 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.331899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.332009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.332146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.332283 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.332403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.332507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.332603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.332729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.332866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.332995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.333127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.333262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.333262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.333501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.333624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.333702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.333790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.333892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.333981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.334057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.334186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.334268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.334422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.334501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.334670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.334751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.334904 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.334985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.335160 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.335237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.335400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.external = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.335537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.335630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.335736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.335862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.external = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.335942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.336092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.336200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.336363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.password = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.336445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.336596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] auth.token = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.336679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.337152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.password = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.337249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.337323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] auth.token = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.337394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.337474 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.337609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.337686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.337759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.337830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.337900 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.337985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.338061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.338722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.338823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.338979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.339060 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.339210 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.339292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.339445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.339524 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.339690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.339772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.339936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.340015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.340177 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.340255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.340413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.340500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.340653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.340731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.340892 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.340972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.341134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.341220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.341390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.341471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.341656 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.341737 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.341912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.342070 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.342190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.342270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.342433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.342513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.342686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.342774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.342926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.343006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.343165 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.343245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.343401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.343487 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.343637 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.343716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.343866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.343944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.344115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.344202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.344341 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.344420 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.344588 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.344668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.344823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.344911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.345075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.345154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.345312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.345390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.345581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.345678 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.345827 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.345944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.346085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.346211 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.346320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.346466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.346568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.346706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.346815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.346950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.347052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.347201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.347283 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.347437 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.347515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.347677 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.347756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.347931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.348011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.348185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.348271 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.348422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.348502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.348653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.348733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.348880 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.348958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.349130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.349216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.349376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.349458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.349650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.349741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.349899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.349978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.350153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.350235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.350402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.350481 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.350654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.350744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.350895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.350975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.351163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.351247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.351402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.351489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.351637 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.351723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.351865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.351943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.352102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.352190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.352341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.352421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.352578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.352656 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.352812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.352899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.353042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.353119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.353287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.353366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.353590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.353687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.353764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.353834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.354570 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.354652 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.354810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.354899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.355043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.355122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.355274 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.355352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.355502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.355588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.355741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.355819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.355976 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.356061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.356214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.356301 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.356449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.356529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.356686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.356772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.356933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.357021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.357166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.357245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.357397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.357475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.357662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.357756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.357917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.357996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.358155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.358235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.358389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.358475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.358625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.358705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.358857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.358942 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.359086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.359174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.359345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.359462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.359550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.359624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.359803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.359891 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.360046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.360125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.360291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.360369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.360529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.360617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.360762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.360843 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.360989 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.361069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.361225 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.361312 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.361457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.361555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.361711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.361791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.361939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.362031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.362185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.362265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.362430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.362509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.362674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.362762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.362917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.362996 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.363146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.363225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.363411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.363528 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.363662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.363735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.363847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.363928 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.364081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.364170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.364315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.364394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.364547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.364626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.364792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.364886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.365030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.365110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.365264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.365344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.365496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.365657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.365762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.365841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.365998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.366078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.366232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.366313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.366468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.366550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.366705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.366785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.366950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.367031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.367185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.367264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.367413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.367491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.367650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.367732 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.367891 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.367977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.368140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.368220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.368382 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.368542 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.368625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.368697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.368850 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.368931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.369098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.369180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.369332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.369413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.369626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.369713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.369868 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.369989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.370104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] role.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.370183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.370336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] role.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.370414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] role.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.370573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.370654 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] role.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.370806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.370884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.371054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.371134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.371302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.371384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.371545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.371624 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.371790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.371869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.372035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.372115 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.372276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.372355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.372522 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.372602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.372755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.372835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.372988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.373066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.373236 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.373315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.373482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.373577 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.373754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.373839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.374002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.374191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.374429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.374510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.374674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.374753 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.374925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.375012 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.375175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.375254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.375426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.375565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.375651 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.375821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.376052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.376132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.376308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.376386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.376552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.376638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.376801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.376898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.377028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.377108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.377261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.377350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.377493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.377642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.377772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.377853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.378026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.378118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.378268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.378349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.378508 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.378587 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.378744 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.378833 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.378974 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.379053 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.379212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.379290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.379446 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.379533 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.379689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.379767 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.379923 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.380002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.380178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.380267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.380424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.380503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.380662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.380741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.380905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.380992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.381148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.381227 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.381386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.381463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.381644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.381737 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.382134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.382223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.382223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.382349 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.382420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.382502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.382576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.382647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.382789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.382871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.383035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.383123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.383328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.383408 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.connection = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.383576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.383665 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.connection = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.383827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.383915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.384066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.384154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.384304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.384383 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.384541 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.384620 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.384775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.384926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.385004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.385077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.385259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.385338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.385493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.385606 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.385759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.385873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.385992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.386071 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.386230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.386318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.386464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.386544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.386712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.386790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.386948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.387036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.387179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.387258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.387412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.387491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.387648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.387734 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.387809 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 18:28:46.387882 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:28:46.388038 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 18:28:46.417995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.418180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.418276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.418344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.418495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.418654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.428644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.428895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.457735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.458025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.458203 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.458366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.510836 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.511527 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.561916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.562147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.562299 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.562540 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.628303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.628489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.628596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.628679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.628755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.628843 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.628955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.629039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.629175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.629260 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.629393 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.629479 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.653733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.653927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.654061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.654203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.654393 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.654562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.654692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.654827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.654970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.655106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.655232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.655358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.655485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.655611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.666054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.666214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.679014 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.679462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.679594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.679750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.679880 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.680008 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.680164 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.680243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.680317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.680399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.680475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.680548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.680620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.680697 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:28:46.698144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.698266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.698332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 18:28:46.698437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 18:28:46.698681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.698780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.698886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 18:28:46.698970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 18:28:46.699167 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.699249 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.699433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.699525 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.699695 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.699777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.699953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.700033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.711394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.711502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.711669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.711754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.711942 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.712025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.712228 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.712315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.719719 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.719821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.722839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.722982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.723147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.723235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.723434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.723517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.723722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.723807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.724009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.724090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.724274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.724355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.724548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.724631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.724825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.724906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.725097 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.725186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.725374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.725454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.725670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.725754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.748960 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.749573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.750532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.750826 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.751073 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.751363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.755420 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.755710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.755946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.756073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.756233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.756523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.766733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.767084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.768677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.768971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.773178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.773473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.773792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.774081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.774163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.774378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.774671 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.774765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.774985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.775175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.775256 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.775674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.776743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.777251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.777779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.778266 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.792350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.792634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.796651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.797600 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.797891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.799817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.800114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.801887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.804741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.805319 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.805649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.808453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.810582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.811621 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.811927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.812226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.812517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.812802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.813105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.813785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.817058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.817367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.817689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.817997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.818280 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.818565 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.829660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.829957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.830229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.830501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.835122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.835419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.835692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.835962 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.837374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.838994 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.839282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.842112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.842406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.842703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.842963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.843056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.843281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.844540 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.844824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.847665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.847960 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.848263 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.848570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.848856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.854674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.854964 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.855250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.855537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.855823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.856102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.856382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.860230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.860519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.860804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.861089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.861376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.861688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.861974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.873557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.876497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.879071 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.879441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.881015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.882034 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.882533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.882829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.883122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.884996 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.887170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.888655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.888951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.889239 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.890047 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.891653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.894603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.894901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.895195 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.895471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.895710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.897300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.900247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.900549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.900869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.901161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.903476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.903777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.904061 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.909174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.909473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.909784 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.910904 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.911196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.911483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.911766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.912049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.912335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.912612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.912882 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.913274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.913582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.913864 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.914177 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.914487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.914725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.916664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:28:46.916951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.917233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.917514 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.917816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.918104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.918384 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.918676 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.919061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.919339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.919617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.919900 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.920170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.920443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:28:46.949046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56376efcf9c0 pid: 80446 (default app) Aug 09 18:28:46.952430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d020658-8dbe-4351-91cb-0ba45b85ed7e None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:46.952721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d020658-8dbe-4351-91cb-0ba45b85ed7e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:46.952993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d020658-8dbe-4351-91cb-0ba45b85ed7e None None] PATH_INFO: `/v3` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:46.955050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56376efcf9c0 pid: 80445 (default app) Aug 09 18:28:46.957375 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Tue Aug 9 18:28:46 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:28:55.561436 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c0a33-b73f-415c-8bec-aebb455035a6 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:55.562011 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c0a33-b73f-415c-8bec-aebb455035a6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:55.562011 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c0a33-b73f-415c-8bec-aebb455035a6 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:55.566472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:28:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:28:55.571967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b89131-795c-41e3-8132-19d9c7d29a24 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:55.572407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b89131-795c-41e3-8132-19d9c7d29a24 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:55.572499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b89131-795c-41e3-8132-19d9c7d29a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:55.642305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: INFO dbcounter [None req-08b89131-795c-41e3-8132-19d9c7d29a24 None None] Registered counter for database keystone Aug 09 18:28:55.663534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG oslo_db.sqlalchemy.engines [None req-08b89131-795c-41e3-8132-19d9c7d29a24 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80446) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 18:28:55.670439 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writer thread running {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 18:28:55.848912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG passlib.handlers.bcrypt [None req-08b89131-795c-41e3-8132-19d9c7d29a24 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80446) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 18:28:55.849237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG passlib.handlers.bcrypt [None req-08b89131-795c-41e3-8132-19d9c7d29a24 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80446) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 18:28:55.905795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-08b89131-795c-41e3-8132-19d9c7d29a24 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:28:55.907430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-08b89131-795c-41e3-8132-19d9c7d29a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:28:56.040681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:28:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 18:28:56.176359 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c0a33-b73f-415c-8bec-aebb455035a6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:28:56.247208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: INFO dbcounter [None req-6a7c0a33-b73f-415c-8bec-aebb455035a6 None None] Registered counter for database keystone Aug 09 18:28:56.266941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG oslo_db.sqlalchemy.engines [None req-6a7c0a33-b73f-415c-8bec-aebb455035a6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80445) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 18:28:56.273433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writer thread running {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 18:28:56.392445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c0a33-b73f-415c-8bec-aebb455035a6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:28:56.396730 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e0694-2885-40d3-8329-069929f25fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:28:56.397353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e0694-2885-40d3-8329-069929f25fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:56.397668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e0694-2885-40d3-8329-069929f25fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:56.397949 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e0694-2885-40d3-8329-069929f25fe2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:56.406563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e0694-2885-40d3-8329-069929f25fe2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:28:56.460029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: WARNING keystone.common.rbac_enforcer.enforcer [None req-843e0694-2885-40d3-8329-069929f25fe2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 18:28:56.460286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e0694-2885-40d3-8329-069929f25fe2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:28:56.470956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Tue Aug 9 18:28:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 18:28:56.477098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b89131-795c-41e3-8132-19d9c7d29a24 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:28:56.492276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b89131-795c-41e3-8132-19d9c7d29a24 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:28:56.497515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4e8b4-5451-4853-9491-a846051d8c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:28:56.498686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4e8b4-5451-4853-9491-a846051d8c03 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:56.498972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4e8b4-5451-4853-9491-a846051d8c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:56.499250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4e8b4-5451-4853-9491-a846051d8c03 admin admin] PATH_INFO: `/v3/projects/ee70c279239f4e719fad35784db1a619` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:56.500702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4e8b4-5451-4853-9491-a846051d8c03 admin admin] RBAC: Authorizing `identity:get_project(project_id=ee70c279239f4e719fad35784db1a619)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:28:56.550300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d5f4e8b4-5451-4853-9491-a846051d8c03 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 18:28:56.550639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4e8b4-5451-4853-9491-a846051d8c03 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:28:56.555675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Tue Aug 9 18:28:56 2022] GET /identity/v3/projects/ee70c279239f4e719fad35784db1a619 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:28:58.534930 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d7d32-53bb-49e8-8a14-9b7921e6249a None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:58.535437 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d7d32-53bb-49e8-8a14-9b7921e6249a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:58.535437 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d7d32-53bb-49e8-8a14-9b7921e6249a None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:58.539628 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:28:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:28:58.545690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d372d14-b3a0-4481-a768-71d2236ee28d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:58.546230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d372d14-b3a0-4481-a768-71d2236ee28d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:58.546230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d372d14-b3a0-4481-a768-71d2236ee28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:58.568243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7d372d14-b3a0-4481-a768-71d2236ee28d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:28:58.569806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7d372d14-b3a0-4481-a768-71d2236ee28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:28:58.577682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:28:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 18:28:58.603881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d7d32-53bb-49e8-8a14-9b7921e6249a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:28:58.616319 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d7d32-53bb-49e8-8a14-9b7921e6249a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:28:58.621253 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb8ed5-6418-4f47-a7f2-a4c2bb770607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:28:58.621871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb8ed5-6418-4f47-a7f2-a4c2bb770607 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:58.622153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb8ed5-6418-4f47-a7f2-a4c2bb770607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:58.622424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb8ed5-6418-4f47-a7f2-a4c2bb770607 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:58.629919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb8ed5-6418-4f47-a7f2-a4c2bb770607 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:28:58.631673 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb8ed5-6418-4f47-a7f2-a4c2bb770607 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:28:58.640167 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Tue Aug 9 18:28:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:28:58.645999 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d372d14-b3a0-4481-a768-71d2236ee28d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:28:58.655089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d372d14-b3a0-4481-a768-71d2236ee28d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:28:58.659682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa00eda-8405-449c-9a08-5af179acd5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:28:58.660303 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa00eda-8405-449c-9a08-5af179acd5df admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:28:58.660587 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa00eda-8405-449c-9a08-5af179acd5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:28:58.660860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa00eda-8405-449c-9a08-5af179acd5df admin admin] PATH_INFO: `/v3/users/7a379f0e1f1f4c9c884081f682310ad9` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:28:58.662279 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa00eda-8405-449c-9a08-5af179acd5df admin admin] RBAC: Authorizing `identity:get_user(user_id=7a379f0e1f1f4c9c884081f682310ad9)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:28:58.664114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa00eda-8405-449c-9a08-5af179acd5df admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:28:58.665716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Tue Aug 9 18:28:58 2022] GET /identity/v3/users/7a379f0e1f1f4c9c884081f682310ad9 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:00.747636 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ac60d-f22c-4562-b757-36505c014787 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:00.748160 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ac60d-f22c-4562-b757-36505c014787 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:00.748160 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ac60d-f22c-4562-b757-36505c014787 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:00.749506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:00.754715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364161e6-193b-401d-8c27-be65db746775 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:00.755136 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364161e6-193b-401d-8c27-be65db746775 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:00.755272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364161e6-193b-401d-8c27-be65db746775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:00.775465 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-364161e6-193b-401d-8c27-be65db746775 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:00.777237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-364161e6-193b-401d-8c27-be65db746775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:00.781965 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce4fdf-a6ac-4689-85a1-bff5e6b6dc71 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:00.782249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce4fdf-a6ac-4689-85a1-bff5e6b6dc71 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:00.782526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce4fdf-a6ac-4689-85a1-bff5e6b6dc71 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:00.783839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:29:00.788821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc51ab5-46cd-4049-8624-95c8195c16ba None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:00.789106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc51ab5-46cd-4049-8624-95c8195c16ba None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:00.789385 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc51ab5-46cd-4049-8624-95c8195c16ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:00.837887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG passlib.handlers.bcrypt [None req-bfc51ab5-46cd-4049-8624-95c8195c16ba None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80445) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 18:29:00.838307 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG passlib.handlers.bcrypt [None req-bfc51ab5-46cd-4049-8624-95c8195c16ba None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80445) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 18:29:00.856923 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:00.879030 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364161e6-193b-401d-8c27-be65db746775 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:00.883911 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-bfc51ab5-46cd-4049-8624-95c8195c16ba None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:00.887233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bfc51ab5-46cd-4049-8624-95c8195c16ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:00.904190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:00.984014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364161e6-193b-401d-8c27-be65db746775 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:00.993870 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:01.020889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc51ab5-46cd-4049-8624-95c8195c16ba None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:01.022649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5029253-7a08-49f7-bdd9-e72b850b04cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:01.023281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5029253-7a08-49f7-bdd9-e72b850b04cb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:01.023560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5029253-7a08-49f7-bdd9-e72b850b04cb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:01.023829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5029253-7a08-49f7-bdd9-e72b850b04cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:01.035814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5029253-7a08-49f7-bdd9-e72b850b04cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-a5029253-7a08-49f7-bdd9-e72b850b04cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:01.044598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:01.050735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:01.051680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:29:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:01.052493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5029253-7a08-49f7-bdd9-e72b850b04cb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:01.126902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc51ab5-46cd-4049-8624-95c8195c16ba None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:01.129248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:01.138434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:01.157100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5029253-7a08-49f7-bdd9-e72b850b04cb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:01.164924 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdc908-a4d6-4223-85d0-9a737407c268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:01.165830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdc908-a4d6-4223-85d0-9a737407c268 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:01.166120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdc908-a4d6-4223-85d0-9a737407c268 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:01.166397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdc908-a4d6-4223-85d0-9a737407c268 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:01.193485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b0c1c-f800-4b9b-abd5-961ee7ba706c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:01.194120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b0c1c-f800-4b9b-abd5-961ee7ba706c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:01.194405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b0c1c-f800-4b9b-abd5-961ee7ba706c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:01.194684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b0c1c-f800-4b9b-abd5-961ee7ba706c None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:01.195354 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b0c1c-f800-4b9b-abd5-961ee7ba706c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:01.195660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b0c1c-f800-4b9b-abd5-961ee7ba706c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:01.197067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b0c1c-f800-4b9b-abd5-961ee7ba706c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:01.209485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:01.215071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b0c1c-f800-4b9b-abd5-961ee7ba706c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:01.238621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdc908-a4d6-4223-85d0-9a737407c268 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:01.240189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdc908-a4d6-4223-85d0-9a737407c268 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:01.351681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 18:29:01.357369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdc908-a4d6-4223-85d0-9a737407c268 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:01.360090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:01.423312 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b0c1c-f800-4b9b-abd5-961ee7ba706c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:01.457858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdc908-a4d6-4223-85d0-9a737407c268 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:01.458073 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88198316-d7d1-4dad-8e9d-90f8049cdc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:01.458644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88198316-d7d1-4dad-8e9d-90f8049cdc4f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:01.458929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88198316-d7d1-4dad-8e9d-90f8049cdc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:01.459210 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88198316-d7d1-4dad-8e9d-90f8049cdc4f None admin] PATH_INFO: `/v3/users/7a379f0e1f1f4c9c884081f682310ad9` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:01.460349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88198316-d7d1-4dad-8e9d-90f8049cdc4f None admin] RBAC: Authorizing `identity:get_user(user_id=7a379f0e1f1f4c9c884081f682310ad9)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:01.461835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88198316-d7d1-4dad-8e9d-90f8049cdc4f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:01.463440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:01 2022] GET /identity/v3/users/7a379f0e1f1f4c9c884081f682310ad9 => generated 247 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:01.468880 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88198316-d7d1-4dad-8e9d-90f8049cdc4f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:01.484330 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:01.492047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20320c-380c-48f1-b9f5-6fb594b259c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:01.496587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20320c-380c-48f1-b9f5-6fb594b259c9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:01.496587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20320c-380c-48f1-b9f5-6fb594b259c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:01.496587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20320c-380c-48f1-b9f5-6fb594b259c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:01.512762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20320c-380c-48f1-b9f5-6fb594b259c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:01.514272 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20320c-380c-48f1-b9f5-6fb594b259c9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-5c20320c-380c-48f1-b9f5-6fb594b259c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:29:01.532830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:01.533806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:29:01.534595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:29:01.534595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:29:01.534595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:01.534595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:01.539722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20320c-380c-48f1-b9f5-6fb594b259c9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:01.570626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88198316-d7d1-4dad-8e9d-90f8049cdc4f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:01.596881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:01.610969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c0709-98a5-4e82-a345-3ce63a593ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:01.610969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c0709-98a5-4e82-a345-3ce63a593ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:01.610969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c0709-98a5-4e82-a345-3ce63a593ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:01.610969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c0709-98a5-4e82-a345-3ce63a593ce0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:01.611370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c0709-98a5-4e82-a345-3ce63a593ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:01.611370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c0709-98a5-4e82-a345-3ce63a593ce0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:01.611370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:01.615370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c0709-98a5-4e82-a345-3ce63a593ce0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:01.641606 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20320c-380c-48f1-b9f5-6fb594b259c9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:01.675444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890d6cb-c24d-4eac-897b-c33db56644a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:01.676145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890d6cb-c24d-4eac-897b-c33db56644a3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:01.676429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890d6cb-c24d-4eac-897b-c33db56644a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:01.676701 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890d6cb-c24d-4eac-897b-c33db56644a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:01.677350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890d6cb-c24d-4eac-897b-c33db56644a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:01.677734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890d6cb-c24d-4eac-897b-c33db56644a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:01.679082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890d6cb-c24d-4eac-897b-c33db56644a3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:01.696902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:01.697183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 10/20] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:01.717694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c0709-98a5-4e82-a345-3ce63a593ce0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:01.756942 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c18b26-5191-4fcd-a88c-21f4e64c1d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:01.757641 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c18b26-5191-4fcd-a88c-21f4e64c1d09 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:01.757862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c18b26-5191-4fcd-a88c-21f4e64c1d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:01.758141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c18b26-5191-4fcd-a88c-21f4e64c1d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:01.758895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c18b26-5191-4fcd-a88c-21f4e64c1d09 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.domain.id=default, user.id=7a379f0e1f1f4c9c884081f682310ad9) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:01.759207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c18b26-5191-4fcd-a88c-21f4e64c1d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:01.760716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c18b26-5191-4fcd-a88c-21f4e64c1d09 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:01.768513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 11/21] 10.222.0.57 () {56 vars in 1322 bytes} [Tue Aug 9 18:29:01 2022] GET /identity/v3/role_assignments?user.id=7a379f0e1f1f4c9c884081f682310ad9&scope.domain.id=default&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:04.031801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d1d7c-02ee-4780-9904-4b456ddab894 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:04.032316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d1d7c-02ee-4780-9904-4b456ddab894 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:04.032316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d1d7c-02ee-4780-9904-4b456ddab894 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:04.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 11/22] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:04.039065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a2241-1d0c-4fbd-afd5-1ec250c19c8d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:04.039494 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a2241-1d0c-4fbd-afd5-1ec250c19c8d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:04.039617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a2241-1d0c-4fbd-afd5-1ec250c19c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:04.070726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-281a2241-1d0c-4fbd-afd5-1ec250c19c8d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:04.071931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-281a2241-1d0c-4fbd-afd5-1ec250c19c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:04.173990 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 12/23] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:04.208280 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d1d7c-02ee-4780-9904-4b456ddab894 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:04.342226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d1d7c-02ee-4780-9904-4b456ddab894 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:04.365075 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:04.370978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c41a2-6fc6-4891-b343-edcfe503104f None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:04.371556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c41a2-6fc6-4891-b343-edcfe503104f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:04.372039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c41a2-6fc6-4891-b343-edcfe503104f None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:04.373613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 13/24] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:29:04.379029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10639410-190f-4a8c-b082-19a479861bf5 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:04.379636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10639410-190f-4a8c-b082-19a479861bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:04.383893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10639410-190f-4a8c-b082-19a479861bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:04.390269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420493a-d5c9-4732-acae-f796f52ba85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:04.390915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420493a-d5c9-4732-acae-f796f52ba85e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:04.391197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420493a-d5c9-4732-acae-f796f52ba85e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:04.391470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420493a-d5c9-4732-acae-f796f52ba85e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:04.406540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-10639410-190f-4a8c-b082-19a479861bf5 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:04.408067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-10639410-190f-4a8c-b082-19a479861bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:04.417547 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420493a-d5c9-4732-acae-f796f52ba85e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-1420493a-d5c9-4732-acae-f796f52ba85e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:04.418883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:04.419710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:04.420445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 12/25] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:29:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:04.426448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420493a-d5c9-4732-acae-f796f52ba85e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:04.557335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 14/26] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:04.590625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10639410-190f-4a8c-b082-19a479861bf5 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:04.595051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:04.736143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420493a-d5c9-4732-acae-f796f52ba85e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:04.822288 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe3490-a51f-4604-9f00-38807f079750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:04.822957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe3490-a51f-4604-9f00-38807f079750 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:04.823244 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe3490-a51f-4604-9f00-38807f079750 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:04.823519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe3490-a51f-4604-9f00-38807f079750 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:04.824214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe3490-a51f-4604-9f00-38807f079750 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:04.824526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe3490-a51f-4604-9f00-38807f079750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:04.826670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe3490-a51f-4604-9f00-38807f079750 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:04.837771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10639410-190f-4a8c-b082-19a479861bf5 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:04.851890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 13/27] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:04.853089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6542e-4678-469f-9ac7-24e23f34e035 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:04.853383 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6542e-4678-469f-9ac7-24e23f34e035 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:04.853705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6542e-4678-469f-9ac7-24e23f34e035 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:04.855667 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 14/28] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:04.862451 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6542e-4678-469f-9ac7-24e23f34e035 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:04.884450 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927a09a-6fed-4a5e-9e0f-d05e04287338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:04.885151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927a09a-6fed-4a5e-9e0f-d05e04287338 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:04.885435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927a09a-6fed-4a5e-9e0f-d05e04287338 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:04.885730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927a09a-6fed-4a5e-9e0f-d05e04287338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:04.910483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927a09a-6fed-4a5e-9e0f-d05e04287338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:04.912036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927a09a-6fed-4a5e-9e0f-d05e04287338 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:04.916321 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-8927a09a-6fed-4a5e-9e0f-d05e04287338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:29:04.935246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:04.938855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:29:04.939657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:29:04.939657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:29:04.939657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:04.939657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 15/29] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:04.939657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c15946-acb0-45e5-824c-9dcd8bd79547 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:04.939657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c15946-acb0-45e5-824c-9dcd8bd79547 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:04.940274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c15946-acb0-45e5-824c-9dcd8bd79547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:04.961358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-63c15946-acb0-45e5-824c-9dcd8bd79547 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:04.963838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-63c15946-acb0-45e5-824c-9dcd8bd79547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:05.058767 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:05.067798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6542e-4678-469f-9ac7-24e23f34e035 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:05.093153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 16/30] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:05.096261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c15946-acb0-45e5-824c-9dcd8bd79547 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:05.134045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecc220-0463-4c2f-bb4c-d5586e26e7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:05.134598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecc220-0463-4c2f-bb4c-d5586e26e7cb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:05.134874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecc220-0463-4c2f-bb4c-d5586e26e7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:05.135750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecc220-0463-4c2f-bb4c-d5586e26e7cb None admin] PATH_INFO: `/v3/users/7a379f0e1f1f4c9c884081f682310ad9` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:05.138147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ecc220-0463-4c2f-bb4c-d5586e26e7cb None admin] RBAC: Authorizing `identity:get_user(user_id=7a379f0e1f1f4c9c884081f682310ad9)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:05.139611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ecc220-0463-4c2f-bb4c-d5586e26e7cb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:05.141358 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 15/31] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:04 2022] GET /identity/v3/users/7a379f0e1f1f4c9c884081f682310ad9 => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:05.146906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480dc6a5-d5ee-4a7f-91b7-3c9ac0b1173a None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:05.147191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480dc6a5-d5ee-4a7f-91b7-3c9ac0b1173a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:05.147467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480dc6a5-d5ee-4a7f-91b7-3c9ac0b1173a None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:05.148773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 16/32] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:05.149563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480dc6a5-d5ee-4a7f-91b7-3c9ac0b1173a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:05.254004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c15946-acb0-45e5-824c-9dcd8bd79547 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:05.265321 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480dc6a5-d5ee-4a7f-91b7-3c9ac0b1173a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:05.274017 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:05.297107 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c660271-d9f8-42fb-8903-9addd9b3e80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:05.297800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c660271-d9f8-42fb-8903-9addd9b3e80e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:05.298085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c660271-d9f8-42fb-8903-9addd9b3e80e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:05.298437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c660271-d9f8-42fb-8903-9addd9b3e80e None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:05.299097 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c660271-d9f8-42fb-8903-9addd9b3e80e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:05.299400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c660271-d9f8-42fb-8903-9addd9b3e80e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:05.300787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c660271-d9f8-42fb-8903-9addd9b3e80e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:05.308524 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75d742-6bcb-45b4-91b2-01346d745f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:05.309225 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75d742-6bcb-45b4-91b2-01346d745f14 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:05.309600 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75d742-6bcb-45b4-91b2-01346d745f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:05.309946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75d742-6bcb-45b4-91b2-01346d745f14 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:05.310680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75d742-6bcb-45b4-91b2-01346d745f14 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:05.312206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75d742-6bcb-45b4-91b2-01346d745f14 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:05.320685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 17/33] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:05.321679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c660271-d9f8-42fb-8903-9addd9b3e80e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:05.375890 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:05.433007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c660271-d9f8-42fb-8903-9addd9b3e80e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:05.470067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e2214-0daf-47cf-964a-b07e3d2416df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:05.470702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e2214-0daf-47cf-964a-b07e3d2416df None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:05.470988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e2214-0daf-47cf-964a-b07e3d2416df None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:05.471264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e2214-0daf-47cf-964a-b07e3d2416df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:05.472398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e2214-0daf-47cf-964a-b07e3d2416df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:05.473957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e2214-0daf-47cf-964a-b07e3d2416df None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:05.480499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 18/34] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:05.480499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b9d0d-d3ef-4598-92d2-6303bd4278f3 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:05.480499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b9d0d-d3ef-4598-92d2-6303bd4278f3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:05.480499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b9d0d-d3ef-4598-92d2-6303bd4278f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:05.496100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-485b9d0d-d3ef-4598-92d2-6303bd4278f3 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:05.497297 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-485b9d0d-d3ef-4598-92d2-6303bd4278f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:05.538477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 17/35] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Aug 9 18:29:05 2022] POST /identity/v3/roles => generated 225 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:29:05.550759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75d742-6bcb-45b4-91b2-01346d745f14 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:05.574953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 19/36] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:05.576385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363309-51bf-4fc8-9199-af816078c212 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:05.576682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363309-51bf-4fc8-9199-af816078c212 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:05.576961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363309-51bf-4fc8-9199-af816078c212 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:05.578638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 20/37] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:05.579196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363309-51bf-4fc8-9199-af816078c212 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:05.635908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:05.679899 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75d742-6bcb-45b4-91b2-01346d745f14 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:05.703163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363309-51bf-4fc8-9199-af816078c212 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:05.734547 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b0ab2-0f1a-4c2c-bbb3-eeef2024b989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:05.735175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b0ab2-0f1a-4c2c-bbb3-eeef2024b989 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:05.735464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b0ab2-0f1a-4c2c-bbb3-eeef2024b989 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:05.735740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b0ab2-0f1a-4c2c-bbb3-eeef2024b989 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:05.736403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b0ab2-0f1a-4c2c-bbb3-eeef2024b989 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:05.738027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b0ab2-0f1a-4c2c-bbb3-eeef2024b989 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:05.746458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff0bb1-0034-4b89-9713-89a80368b258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:05.747423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff0bb1-0034-4b89-9713-89a80368b258 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:05.747949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff0bb1-0034-4b89-9713-89a80368b258 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:05.748469 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff0bb1-0034-4b89-9713-89a80368b258 None admin] PATH_INFO: `/v3/domains/default/users/7a379f0e1f1f4c9c884081f682310ad9/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:05.750668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff0bb1-0034-4b89-9713-89a80368b258 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7a379f0e1f1f4c9c884081f682310ad9, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:05.752381 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff0bb1-0034-4b89-9713-89a80368b258 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:05.909752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 18/38] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:29:05 2022] POST /identity/v3/projects => generated 301 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:29:05.911049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b5008-5624-497b-9eda-8cb5ce5df0ef None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:05.911338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b5008-5624-497b-9eda-8cb5ce5df0ef None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:05.911615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b5008-5624-497b-9eda-8cb5ce5df0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:05.931915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ab9b5008-5624-497b-9eda-8cb5ce5df0ef None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:05.933627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ab9b5008-5624-497b-9eda-8cb5ce5df0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:05.950884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:05.977653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 21/39] 10.222.0.57 () {58 vars in 1361 bytes} [Tue Aug 9 18:29:05 2022] PUT /identity/v3/domains/default/users/7a379f0e1f1f4c9c884081f682310ad9/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:29:05.977653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff0bb1-0034-4b89-9713-89a80368b258 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:06.030388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 19/40] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:06.032379 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2613c-b94d-4d69-a9f0-9791e66937c2 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:06.034501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2613c-b94d-4d69-a9f0-9791e66937c2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:06.036187 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2613c-b94d-4d69-a9f0-9791e66937c2 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:06.042432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 20/41] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:06 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:06.045625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac3686-3272-4737-aafd-6985e403e8d7 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:06.046136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac3686-3272-4737-aafd-6985e403e8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:06.046516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac3686-3272-4737-aafd-6985e403e8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:06.073637 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f4ac3686-3272-4737-aafd-6985e403e8d7 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:06.075938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f4ac3686-3272-4737-aafd-6985e403e8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:06.131762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff0bb1-0034-4b89-9713-89a80368b258 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:06.136217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:06.180398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fc958-8211-4d2b-83d1-2a856aed5871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:06.181081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fc958-8211-4d2b-83d1-2a856aed5871 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:06.181366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fc958-8211-4d2b-83d1-2a856aed5871 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:06.181702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fc958-8211-4d2b-83d1-2a856aed5871 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:06.182412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fc958-8211-4d2b-83d1-2a856aed5871 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:06.182514 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 21/42] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:06.183296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac3686-3272-4737-aafd-6985e403e8d7 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:06.184158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fc958-8211-4d2b-83d1-2a856aed5871 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:06.249384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 22/43] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Aug 9 18:29:05 2022] POST /identity/v3/roles => generated 219 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:29:06.250210 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fc958-8211-4d2b-83d1-2a856aed5871 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:06.289499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:06.318244 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac3686-3272-4737-aafd-6985e403e8d7 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:06.361826 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf1c58-6226-4387-a3b0-dd57432786d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:06.362554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf1c58-6226-4387-a3b0-dd57432786d1 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:06.362920 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf1c58-6226-4387-a3b0-dd57432786d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:06.363269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf1c58-6226-4387-a3b0-dd57432786d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:06.364021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf1c58-6226-4387-a3b0-dd57432786d1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:06.365747 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf1c58-6226-4387-a3b0-dd57432786d1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:06.380144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fc958-8211-4d2b-83d1-2a856aed5871 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:06.423102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b088e-03ef-4d24-8ec3-7eb51277b602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:06.423750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b088e-03ef-4d24-8ec3-7eb51277b602 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:06.424037 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b088e-03ef-4d24-8ec3-7eb51277b602 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:06.424318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b088e-03ef-4d24-8ec3-7eb51277b602 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:06.425587 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b088e-03ef-4d24-8ec3-7eb51277b602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:06.427142 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b088e-03ef-4d24-8ec3-7eb51277b602 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:06.428970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 23/44] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:06.435476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b088e-03ef-4d24-8ec3-7eb51277b602 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:06.455674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 22/45] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Aug 9 18:29:06 2022] POST /identity/v3/roles => generated 223 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:29:06.471930 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:06.559679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b088e-03ef-4d24-8ec3-7eb51277b602 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:06.599556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7f3b5-6323-4208-91e3-719bfc4d99fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:06.600187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7f3b5-6323-4208-91e3-719bfc4d99fc None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:06.600471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7f3b5-6323-4208-91e3-719bfc4d99fc None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:06.600749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7f3b5-6323-4208-91e3-719bfc4d99fc None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:06.601406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7f3b5-6323-4208-91e3-719bfc4d99fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:06.602966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7f3b5-6323-4208-91e3-719bfc4d99fc None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:06.733220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 24/46] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:29:06 2022] POST /identity/v3/projects => generated 312 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:29:08.105137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcf224-988a-4667-add1-294c560a79bc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:08.105728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcf224-988a-4667-add1-294c560a79bc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:08.106050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcf224-988a-4667-add1-294c560a79bc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:08.107503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 23/47] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:08.113476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26693d7-2361-4e86-9341-c593e665d3f2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:08.113955 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26693d7-2361-4e86-9341-c593e665d3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:08.114112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26693d7-2361-4e86-9341-c593e665d3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:08.136523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a26693d7-2361-4e86-9341-c593e665d3f2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:08.138076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a26693d7-2361-4e86-9341-c593e665d3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:08.210416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 25/48] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:08.235869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcf224-988a-4667-add1-294c560a79bc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:08.341447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcf224-988a-4667-add1-294c560a79bc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:08.375696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dc636-3faa-49f9-aed0-a9c0882ad77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:08.376247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dc636-3faa-49f9-aed0-a9c0882ad77b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:08.376526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dc636-3faa-49f9-aed0-a9c0882ad77b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:08.376800 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dc636-3faa-49f9-aed0-a9c0882ad77b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:08.383742 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:08.387762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3dc636-3faa-49f9-aed0-a9c0882ad77b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-fa3dc636-3faa-49f9-aed0-a9c0882ad77b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:08.392476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:08.393286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:08.394059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 24/49] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:29:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:08.399434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26693d7-2361-4e86-9341-c593e665d3f2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:08.415039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:08.514176 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26693d7-2361-4e86-9341-c593e665d3f2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:08.550038 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a09f6-f9ce-4c0f-92f1-5c5957177458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:08.550780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a09f6-f9ce-4c0f-92f1-5c5957177458 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:08.551149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a09f6-f9ce-4c0f-92f1-5c5957177458 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:08.551515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a09f6-f9ce-4c0f-92f1-5c5957177458 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:08.552287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a09f6-f9ce-4c0f-92f1-5c5957177458 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:08.552682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a09f6-f9ce-4c0f-92f1-5c5957177458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:08.554511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a09f6-f9ce-4c0f-92f1-5c5957177458 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:08.563827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 26/50] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:08.569937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dc636-3faa-49f9-aed0-a9c0882ad77b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:08.668706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dc636-3faa-49f9-aed0-a9c0882ad77b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:08.702809 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc612d-b30b-417a-bcdf-c6fa8eea852e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:08.703438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc612d-b30b-417a-bcdf-c6fa8eea852e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:08.703725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc612d-b30b-417a-bcdf-c6fa8eea852e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:08.704001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc612d-b30b-417a-bcdf-c6fa8eea852e None admin] PATH_INFO: `/v3/users/7a379f0e1f1f4c9c884081f682310ad9` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:08.705130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc612d-b30b-417a-bcdf-c6fa8eea852e None admin] RBAC: Authorizing `identity:get_user(user_id=7a379f0e1f1f4c9c884081f682310ad9)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:08.706620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc612d-b30b-417a-bcdf-c6fa8eea852e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:08.708200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 25/51] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:08 2022] GET /identity/v3/users/7a379f0e1f1f4c9c884081f682310ad9 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:08.714302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a09f6-f9ce-4c0f-92f1-5c5957177458 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:08.782907 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992cd92-9125-47b3-b86f-4d22113ceadc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:08.783262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992cd92-9125-47b3-b86f-4d22113ceadc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:08.783480 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992cd92-9125-47b3-b86f-4d22113ceadc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:08.784839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 26/52] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:29:08.789761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f05a94-239b-44aa-a2db-d429a1c92f93 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:08.790056 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f05a94-239b-44aa-a2db-d429a1c92f93 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:08.790338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f05a94-239b-44aa-a2db-d429a1c92f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:08.790742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:08.813816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a09f6-f9ce-4c0f-92f1-5c5957177458 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:08.817174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c5f05a94-239b-44aa-a2db-d429a1c92f93 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:08.818566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c5f05a94-239b-44aa-a2db-d429a1c92f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:08.854341 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00063837-9258-469e-b25b-8d061f496352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:08.854988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00063837-9258-469e-b25b-8d061f496352 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:08.855272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00063837-9258-469e-b25b-8d061f496352 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:08.856938 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00063837-9258-469e-b25b-8d061f496352 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:08.857330 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00063837-9258-469e-b25b-8d061f496352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:08.858833 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00063837-9258-469e-b25b-8d061f496352 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:08.860422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 27/53] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:08.866216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00063837-9258-469e-b25b-8d061f496352 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:08.887166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:08.895254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 27/54] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:08.918471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f05a94-239b-44aa-a2db-d429a1c92f93 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:08.968267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00063837-9258-469e-b25b-8d061f496352 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:09.004313 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ca02e-15de-4ad3-8a63-9234e2e391ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:09.004958 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ca02e-15de-4ad3-8a63-9234e2e391ac None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:09.005239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ca02e-15de-4ad3-8a63-9234e2e391ac None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:09.005510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ca02e-15de-4ad3-8a63-9234e2e391ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:09.006334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ca02e-15de-4ad3-8a63-9234e2e391ac None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.domain.id=default, user.id=7a379f0e1f1f4c9c884081f682310ad9) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:09.006640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ca02e-15de-4ad3-8a63-9234e2e391ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:09.008203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ca02e-15de-4ad3-8a63-9234e2e391ac None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:09.014801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 28/55] 10.222.0.57 () {56 vars in 1322 bytes} [Tue Aug 9 18:29:08 2022] GET /identity/v3/role_assignments?user.id=7a379f0e1f1f4c9c884081f682310ad9&scope.domain.id=default&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:09.018448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f05a94-239b-44aa-a2db-d429a1c92f93 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:09.056110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ba663-4845-4d45-97a6-df2f91946dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:09.056756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ba663-4845-4d45-97a6-df2f91946dbb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:09.057037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ba663-4845-4d45-97a6-df2f91946dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:09.057309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ba663-4845-4d45-97a6-df2f91946dbb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:09.058497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ba663-4845-4d45-97a6-df2f91946dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:09.059970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ba663-4845-4d45-97a6-df2f91946dbb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:09.061819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 28/56] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:09.068217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ca02e-15de-4ad3-8a63-9234e2e391ac None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:09.080896 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:09.179089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ca02e-15de-4ad3-8a63-9234e2e391ac None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:09.224173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7d163-0688-4706-889b-26a18dc4b82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:09.225049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7d163-0688-4706-889b-26a18dc4b82d None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:09.225543 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7d163-0688-4706-889b-26a18dc4b82d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:09.226123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7d163-0688-4706-889b-26a18dc4b82d None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:09.226828 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7d163-0688-4706-889b-26a18dc4b82d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:09.228926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7d163-0688-4706-889b-26a18dc4b82d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:09.342360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 29/57] 10.222.0.57 () {60 vars in 1140 bytes} [Tue Aug 9 18:29:09 2022] POST /identity/v3/projects => generated 298 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:29:11.345599 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea9a41-cf0f-4ee8-a8d1-68ef724b5546 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:11.346105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea9a41-cf0f-4ee8-a8d1-68ef724b5546 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:11.346207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea9a41-cf0f-4ee8-a8d1-68ef724b5546 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:11.347494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 29/58] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:11.352695 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c95c-e33b-4619-9f56-7cf65b9b5269 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:11.353102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c95c-e33b-4619-9f56-7cf65b9b5269 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:11.353250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c95c-e33b-4619-9f56-7cf65b9b5269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:11.371950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1f63c95c-e33b-4619-9f56-7cf65b9b5269 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:11.373128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1f63c95c-e33b-4619-9f56-7cf65b9b5269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:11.428588 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:11.440654 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 30/59] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:11.464608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea9a41-cf0f-4ee8-a8d1-68ef724b5546 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:11.528503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:11.568242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea9a41-cf0f-4ee8-a8d1-68ef724b5546 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:11.601827 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e67a45-a8a8-45d9-b0c4-b3d0a3042eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:11.602447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e67a45-a8a8-45d9-b0c4-b3d0a3042eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:11.602731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e67a45-a8a8-45d9-b0c4-b3d0a3042eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:11.603005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e67a45-a8a8-45d9-b0c4-b3d0a3042eb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:11.604786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e67a45-a8a8-45d9-b0c4-b3d0a3042eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:11.606255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e67a45-a8a8-45d9-b0c4-b3d0a3042eb6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:11.607823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 30/60] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:11.613453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63c95c-e33b-4619-9f56-7cf65b9b5269 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:11.708291 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63c95c-e33b-4619-9f56-7cf65b9b5269 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:11.741816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b9a11-c365-41d1-a968-38db826645a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:11.742413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b9a11-c365-41d1-a968-38db826645a8 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:11.742699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b9a11-c365-41d1-a968-38db826645a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:11.742979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b9a11-c365-41d1-a968-38db826645a8 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:11.743625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b9a11-c365-41d1-a968-38db826645a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:11.745110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b9a11-c365-41d1-a968-38db826645a8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:11.866524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 31/61] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:29:11 2022] POST /identity/v3/users => generated 275 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:29:14.979860 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6aadfc-69b9-4ac3-93ca-67cadc55d407 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:14.980383 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6aadfc-69b9-4ac3-93ca-67cadc55d407 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:14.980383 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6aadfc-69b9-4ac3-93ca-67cadc55d407 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:14.982044 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 31/62] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:14.987318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e13f5f-1275-4ef3-b989-bc4ca46d03d0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:14.988162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e13f5f-1275-4ef3-b989-bc4ca46d03d0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:14.988634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e13f5f-1275-4ef3-b989-bc4ca46d03d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.014606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-03e13f5f-1275-4ef3-b989-bc4ca46d03d0 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:15.015829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-03e13f5f-1275-4ef3-b989-bc4ca46d03d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:15.090220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:15.123848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 32/63] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:15.163070 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6aadfc-69b9-4ac3-93ca-67cadc55d407 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:15.298687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471ff28-818f-4948-a271-0ba43218dfe7 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.299004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471ff28-818f-4948-a271-0ba43218dfe7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.299244 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471ff28-818f-4948-a271-0ba43218dfe7 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.300610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 33/64] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:29:15.312549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e1744-1735-4703-b361-6564cc569c6b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.312862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e1744-1735-4703-b361-6564cc569c6b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.313107 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e1744-1735-4703-b361-6564cc569c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.319036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6aadfc-69b9-4ac3-93ca-67cadc55d407 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:15.342442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-390e1744-1735-4703-b361-6564cc569c6b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:15.343636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-390e1744-1735-4703-b361-6564cc569c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:15.365068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:15.370842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d50f18-a3bd-4a17-b75b-bc8bdf68dac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:15.371484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d50f18-a3bd-4a17-b75b-bc8bdf68dac9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.371772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d50f18-a3bd-4a17-b75b-bc8bdf68dac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.372049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d50f18-a3bd-4a17-b75b-bc8bdf68dac9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.387179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d50f18-a3bd-4a17-b75b-bc8bdf68dac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-82d50f18-a3bd-4a17-b75b-bc8bdf68dac9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:15.392794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:15.393881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:15.394839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 32/65] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:15.395864 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede1993-7d4d-4c77-82dc-f106b9a9860e None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.396012 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede1993-7d4d-4c77-82dc-f106b9a9860e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.396254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede1993-7d4d-4c77-82dc-f106b9a9860e None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.401593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 33/66] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:15.403059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede1993-7d4d-4c77-82dc-f106b9a9860e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:15.433368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 34/67] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:15.435096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e52e2a-353f-4ae2-8348-5d44a20d4642 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.435579 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e52e2a-353f-4ae2-8348-5d44a20d4642 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.436045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e52e2a-353f-4ae2-8348-5d44a20d4642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.464896 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d8e52e2a-353f-4ae2-8348-5d44a20d4642 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:15.467303 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d8e52e2a-353f-4ae2-8348-5d44a20d4642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:15.550589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 35/68] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:15.551404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e52e2a-353f-4ae2-8348-5d44a20d4642 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:15.590747 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede1993-7d4d-4c77-82dc-f106b9a9860e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:15.590971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:15.633063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a57f0-6f1d-4218-a44d-8ec354fc28d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:15.633716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a57f0-6f1d-4218-a44d-8ec354fc28d1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.634009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a57f0-6f1d-4218-a44d-8ec354fc28d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.634290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a57f0-6f1d-4218-a44d-8ec354fc28d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.634963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a57f0-6f1d-4218-a44d-8ec354fc28d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:15.635271 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a57f0-6f1d-4218-a44d-8ec354fc28d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:15.636652 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a57f0-6f1d-4218-a44d-8ec354fc28d1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:15.648964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 34/69] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:15.650203 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ac0af-b7e0-4e38-aa0b-a1fe27341670 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.650491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ac0af-b7e0-4e38-aa0b-a1fe27341670 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.651303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ac0af-b7e0-4e38-aa0b-a1fe27341670 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.652526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 35/70] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:15.653327 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ac0af-b7e0-4e38-aa0b-a1fe27341670 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:15.667605 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e52e2a-353f-4ae2-8348-5d44a20d4642 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:15.712463 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:15.715623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fed215-e516-4bb6-9acf-08ff4e65606b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:15.716623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fed215-e516-4bb6-9acf-08ff4e65606b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.717064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fed215-e516-4bb6-9acf-08ff4e65606b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.717537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fed215-e516-4bb6-9acf-08ff4e65606b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.734552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fed215-e516-4bb6-9acf-08ff4e65606b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-06fed215-e516-4bb6-9acf-08ff4e65606b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:15.743352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:15.744318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:15.745157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 36/71] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:29:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:15.747150 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a7c3-fa42-40a2-bc60-b4bd5ff99da6 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.747520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a7c3-fa42-40a2-bc60-b4bd5ff99da6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.747813 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a7c3-fa42-40a2-bc60-b4bd5ff99da6 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.753377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 37/72] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:15.753377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050a7c3-fa42-40a2-bc60-b4bd5ff99da6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:15.764574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ac0af-b7e0-4e38-aa0b-a1fe27341670 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:15.799558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba84e3-e954-4441-b01d-5bf418e90617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:15.800280 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba84e3-e954-4441-b01d-5bf418e90617 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.800791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba84e3-e954-4441-b01d-5bf418e90617 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.800791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba84e3-e954-4441-b01d-5bf418e90617 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.811307 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba84e3-e954-4441-b01d-5bf418e90617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-e1ba84e3-e954-4441-b01d-5bf418e90617 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:15.820632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:15.821566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:15.822317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 36/73] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:29:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:15.823881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cab02-f397-40d7-aacc-e883f9a27631 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.823881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cab02-f397-40d7-aacc-e883f9a27631 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.824112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cab02-f397-40d7-aacc-e883f9a27631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.857406 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-293cab02-f397-40d7-aacc-e883f9a27631 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:15.860288 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-293cab02-f397-40d7-aacc-e883f9a27631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:15.881886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:15.889622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050a7c3-fa42-40a2-bc60-b4bd5ff99da6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:15.925110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65a635-7a7f-45bc-92d0-e4a4d6c3ea8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:15.925785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65a635-7a7f-45bc-92d0-e4a4d6c3ea8b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.926346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65a635-7a7f-45bc-92d0-e4a4d6c3ea8b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.926346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65a635-7a7f-45bc-92d0-e4a4d6c3ea8b None admin] PATH_INFO: `/v3/users/2a5cf324b6904a499ca6308498e53335` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.931088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 37/74] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:15.931966 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cab02-f397-40d7-aacc-e883f9a27631 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:15.941067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65a635-7a7f-45bc-92d0-e4a4d6c3ea8b None admin] RBAC: Authorizing `identity:get_user(user_id=2a5cf324b6904a499ca6308498e53335)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:15.942593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65a635-7a7f-45bc-92d0-e4a4d6c3ea8b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:15.944258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 38/75] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:15 2022] GET /identity/v3/users/2a5cf324b6904a499ca6308498e53335 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:15.950663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58909afe-81b4-4890-8445-7084e7b27f90 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:15.950951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58909afe-81b4-4890-8445-7084e7b27f90 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:15.951236 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58909afe-81b4-4890-8445-7084e7b27f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:15.969174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-58909afe-81b4-4890-8445-7084e7b27f90 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:15.970418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-58909afe-81b4-4890-8445-7084e7b27f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:15.981589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:16.038287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cab02-f397-40d7-aacc-e883f9a27631 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:16.038508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 39/76] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:16.050034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58909afe-81b4-4890-8445-7084e7b27f90 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:16.084766 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab2352-fd70-483b-b297-b7557fab8238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:16.085377 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab2352-fd70-483b-b297-b7557fab8238 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:16.085708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab2352-fd70-483b-b297-b7557fab8238 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:16.085982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab2352-fd70-483b-b297-b7557fab8238 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:16.086654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab2352-fd70-483b-b297-b7557fab8238 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:16.086961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab2352-fd70-483b-b297-b7557fab8238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:16.088368 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab2352-fd70-483b-b297-b7557fab8238 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:16.097988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 38/77] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:29:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:16.097988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab2352-fd70-483b-b297-b7557fab8238 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:16.130875 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:16.156791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58909afe-81b4-4890-8445-7084e7b27f90 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:16.201561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd92f9-86d6-4643-95a4-8b4b697043bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:16.202204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd92f9-86d6-4643-95a4-8b4b697043bc None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:16.202547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd92f9-86d6-4643-95a4-8b4b697043bc None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:16.202829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd92f9-86d6-4643-95a4-8b4b697043bc None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:16.203546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd92f9-86d6-4643-95a4-8b4b697043bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:16.203861 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd92f9-86d6-4643-95a4-8b4b697043bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:16.205478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd92f9-86d6-4643-95a4-8b4b697043bc None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:16.206918 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab2352-fd70-483b-b297-b7557fab8238 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:16.214459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:16.220730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 40/78] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:16.221613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd92f9-86d6-4643-95a4-8b4b697043bc None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:16.241733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec73b1-0111-4d5d-a2c8-2ed983f620e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:16.242337 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec73b1-0111-4d5d-a2c8-2ed983f620e9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:16.242617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec73b1-0111-4d5d-a2c8-2ed983f620e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:16.242891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec73b1-0111-4d5d-a2c8-2ed983f620e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:16.313644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec73b1-0111-4d5d-a2c8-2ed983f620e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:16.315108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec73b1-0111-4d5d-a2c8-2ed983f620e9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:16.323223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd92f9-86d6-4643-95a4-8b4b697043bc None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:16.357825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe1ad5-5b36-4ef4-b060-afd82e0972f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:16.358384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe1ad5-5b36-4ef4-b060-afd82e0972f2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:16.358667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe1ad5-5b36-4ef4-b060-afd82e0972f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:16.358942 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe1ad5-5b36-4ef4-b060-afd82e0972f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:16.370100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe1ad5-5b36-4ef4-b060-afd82e0972f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-abfe1ad5-5b36-4ef4-b060-afd82e0972f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:16.375397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:16.376369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:16.377146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 41/79] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:16.377803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe1ad5-5b36-4ef4-b060-afd82e0972f2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:16.387051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:16.420966 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 39/80] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:16.421858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec73b1-0111-4d5d-a2c8-2ed983f620e9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:16.433000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:16.481948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe1ad5-5b36-4ef4-b060-afd82e0972f2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:16.520393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d94329-aee1-4591-b45a-60bed0e8f586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:16.521044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d94329-aee1-4591-b45a-60bed0e8f586 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:16.521324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d94329-aee1-4591-b45a-60bed0e8f586 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:16.521630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d94329-aee1-4591-b45a-60bed0e8f586 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:16.522781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d94329-aee1-4591-b45a-60bed0e8f586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:16.524262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d94329-aee1-4591-b45a-60bed0e8f586 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:16.525909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 42/81] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:16.526683 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d94329-aee1-4591-b45a-60bed0e8f586 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:16.540069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec73b1-0111-4d5d-a2c8-2ed983f620e9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:16.575836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9581e-7475-408c-ad57-d374ccacb517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:16.576458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9581e-7475-408c-ad57-d374ccacb517 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:16.576736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9581e-7475-408c-ad57-d374ccacb517 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:16.577006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9581e-7475-408c-ad57-d374ccacb517 None admin] PATH_INFO: `/v3/users/2a5cf324b6904a499ca6308498e53335` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:16.578139 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9581e-7475-408c-ad57-d374ccacb517 None admin] RBAC: Authorizing `identity:get_user(user_id=2a5cf324b6904a499ca6308498e53335)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:16.579588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9581e-7475-408c-ad57-d374ccacb517 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:16.581167 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 40/82] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:16 2022] GET /identity/v3/users/2a5cf324b6904a499ca6308498e53335 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:16.582101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9581e-7475-408c-ad57-d374ccacb517 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:16.611099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:16.638638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d94329-aee1-4591-b45a-60bed0e8f586 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:16.660021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:16.673780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428c272-bef4-4d50-987e-a62b45574ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:16.674421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428c272-bef4-4d50-987e-a62b45574ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:16.674705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428c272-bef4-4d50-987e-a62b45574ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:16.674985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428c272-bef4-4d50-987e-a62b45574ca2 None admin] PATH_INFO: `/v3/users/7a379f0e1f1f4c9c884081f682310ad9` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:16.676141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428c272-bef4-4d50-987e-a62b45574ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=7a379f0e1f1f4c9c884081f682310ad9)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:16.677651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428c272-bef4-4d50-987e-a62b45574ca2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:16.679523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 43/83] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:16 2022] GET /identity/v3/users/7a379f0e1f1f4c9c884081f682310ad9 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:16.684672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428c272-bef4-4d50-987e-a62b45574ca2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:16.690275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9581e-7475-408c-ad57-d374ccacb517 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:16.725318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e999a-a3fe-45aa-a6f2-c788ff0b6629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:16.726085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e999a-a3fe-45aa-a6f2-c788ff0b6629 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:16.726367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e999a-a3fe-45aa-a6f2-c788ff0b6629 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:16.726645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e999a-a3fe-45aa-a6f2-c788ff0b6629 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:16.727324 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e999a-a3fe-45aa-a6f2-c788ff0b6629 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:16.727652 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e999a-a3fe-45aa-a6f2-c788ff0b6629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:16.729555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e999a-a3fe-45aa-a6f2-c788ff0b6629 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:16.738104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 41/84] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:16.738906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e999a-a3fe-45aa-a6f2-c788ff0b6629 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:16.788411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428c272-bef4-4d50-987e-a62b45574ca2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:16.825477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211584cf-567a-4949-b5b9-d9ff76d29ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:16.826178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211584cf-567a-4949-b5b9-d9ff76d29ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:16.826463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211584cf-567a-4949-b5b9-d9ff76d29ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:16.826745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211584cf-567a-4949-b5b9-d9ff76d29ff1 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:16.827891 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211584cf-567a-4949-b5b9-d9ff76d29ff1 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:16.829375 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211584cf-567a-4949-b5b9-d9ff76d29ff1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:16.831040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 44/85] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:16 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:16.831817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211584cf-567a-4949-b5b9-d9ff76d29ff1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:16.854878 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:16.869792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e999a-a3fe-45aa-a6f2-c788ff0b6629 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:16.899229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aafdec-cb26-45e6-bc17-45bd667c4dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:16.899833 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aafdec-cb26-45e6-bc17-45bd667c4dcb None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:16.900114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aafdec-cb26-45e6-bc17-45bd667c4dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:16.900387 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aafdec-cb26-45e6-bc17-45bd667c4dcb None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:16.901563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aafdec-cb26-45e6-bc17-45bd667c4dcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:16.903021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aafdec-cb26-45e6-bc17-45bd667c4dcb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:16.915490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:16.945069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211584cf-567a-4949-b5b9-d9ff76d29ff1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:16.978363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7d2ab-10b3-4793-9c0e-4628599180f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:16.979627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7d2ab-10b3-4793-9c0e-4628599180f9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:16.979917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7d2ab-10b3-4793-9c0e-4628599180f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:16.980194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7d2ab-10b3-4793-9c0e-4628599180f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:17.015399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 42/86] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:29:16 2022] POST /identity/v3/users => generated 289 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:29:17.030250 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aafdec-cb26-45e6-bc17-45bd667c4dcb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:17.054507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7d2ab-10b3-4793-9c0e-4628599180f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:17.055998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7d2ab-10b3-4793-9c0e-4628599180f9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:17.071801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:17.158929 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aafdec-cb26-45e6-bc17-45bd667c4dcb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:17.187791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 45/87] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:17.187791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7d2ab-10b3-4793-9c0e-4628599180f9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:17.214252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c6f3d-7806-4740-b3ff-3228655c708e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:17.215242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c6f3d-7806-4740-b3ff-3228655c708e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:17.215524 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c6f3d-7806-4740-b3ff-3228655c708e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:17.217608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c6f3d-7806-4740-b3ff-3228655c708e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:17.299353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:17.311731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c6f3d-7806-4740-b3ff-3228655c708e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:17.313484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c6f3d-7806-4740-b3ff-3228655c708e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:17.329926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7d2ab-10b3-4793-9c0e-4628599180f9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:17.345272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:17.377398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd9a4e-1f04-4b00-8841-60058182622e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:17.378369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd9a4e-1f04-4b00-8841-60058182622e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:17.378657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd9a4e-1f04-4b00-8841-60058182622e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:17.379004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd9a4e-1f04-4b00-8841-60058182622e None admin] PATH_INFO: `/v3/users/2a5cf324b6904a499ca6308498e53335` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:17.380174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd9a4e-1f04-4b00-8841-60058182622e None admin] RBAC: Authorizing `identity:get_user(user_id=2a5cf324b6904a499ca6308498e53335)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:17.381909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd9a4e-1f04-4b00-8841-60058182622e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:17.384734 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 46/88] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:17 2022] GET /identity/v3/users/2a5cf324b6904a499ca6308498e53335 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:17.387965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd9a4e-1f04-4b00-8841-60058182622e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:17.445252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 43/89] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:17.445972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c6f3d-7806-4740-b3ff-3228655c708e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:17.499009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd9a4e-1f04-4b00-8841-60058182622e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:17.542155 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c205a63-0a8d-49a5-b066-196eecd17432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:17.543604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c205a63-0a8d-49a5-b066-196eecd17432 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:17.543895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c205a63-0a8d-49a5-b066-196eecd17432 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:17.544179 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c205a63-0a8d-49a5-b066-196eecd17432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:17.544922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c205a63-0a8d-49a5-b066-196eecd17432 None admin] RBAC: Adding query filter params (role.id=08a7c861d84b43d7a4da431f31f3c2d7, scope.project.id=5d18db22ee924d578a2f7079502940eb, user.id=2a5cf324b6904a499ca6308498e53335) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:17.545237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c205a63-0a8d-49a5-b066-196eecd17432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:17.546742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c205a63-0a8d-49a5-b066-196eecd17432 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:17.554802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 47/90] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:17 2022] GET /identity/v3/role_assignments?user.id=2a5cf324b6904a499ca6308498e53335&scope.project.id=5d18db22ee924d578a2f7079502940eb&role.id=08a7c861d84b43d7a4da431f31f3c2d7 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:17.555629 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c205a63-0a8d-49a5-b066-196eecd17432 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:17.578735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:17.611582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c6f3d-7806-4740-b3ff-3228655c708e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:17.637713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:17.687258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c72aa-04fb-42e5-94b6-1a09f1cb4369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:17.688157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c72aa-04fb-42e5-94b6-1a09f1cb4369 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:17.688754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c72aa-04fb-42e5-94b6-1a09f1cb4369 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:17.689189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c72aa-04fb-42e5-94b6-1a09f1cb4369 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:17.693398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c72aa-04fb-42e5-94b6-1a09f1cb4369 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:17.695427 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c72aa-04fb-42e5-94b6-1a09f1cb4369 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:17.698361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 44/91] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:17 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:17.699488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c72aa-04fb-42e5-94b6-1a09f1cb4369 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:17.730013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c205a63-0a8d-49a5-b066-196eecd17432 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:17.802045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ec263-db27-40b4-8b87-364fe05a2844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:17.802669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ec263-db27-40b4-8b87-364fe05a2844 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:17.802953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ec263-db27-40b4-8b87-364fe05a2844 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:17.803230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ec263-db27-40b4-8b87-364fe05a2844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:17.854573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c72aa-04fb-42e5-94b6-1a09f1cb4369 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:17.913735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d2a12-828f-4124-8e75-eab2680a6b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:17.914075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ec263-db27-40b4-8b87-364fe05a2844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:17.914379 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d2a12-828f-4124-8e75-eab2680a6b8d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:17.914668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d2a12-828f-4124-8e75-eab2680a6b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:17.914947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d2a12-828f-4124-8e75-eab2680a6b8d None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:17.916759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ec263-db27-40b4-8b87-364fe05a2844 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:17.922401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d2a12-828f-4124-8e75-eab2680a6b8d None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:17.923895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d2a12-828f-4124-8e75-eab2680a6b8d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:17.925812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 45/92] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:17 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:17.926755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d2a12-828f-4124-8e75-eab2680a6b8d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:17.970033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:17.981214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:18.049939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 48/93] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:18.051081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ec263-db27-40b4-8b87-364fe05a2844 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:18.096741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d2a12-828f-4124-8e75-eab2680a6b8d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:18.159904 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ec263-db27-40b4-8b87-364fe05a2844 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:18.199530 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99bcd0-8733-47bf-b8ab-b36887161754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:18.200374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99bcd0-8733-47bf-b8ab-b36887161754 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:18.200795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99bcd0-8733-47bf-b8ab-b36887161754 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:18.200994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25da273-2c50-44ba-b816-b8295661755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:18.201388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99bcd0-8733-47bf-b8ab-b36887161754 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:18.201656 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25da273-2c50-44ba-b816-b8295661755d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:18.201954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25da273-2c50-44ba-b816-b8295661755d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:18.202235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25da273-2c50-44ba-b816-b8295661755d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:18.202824 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99bcd0-8733-47bf-b8ab-b36887161754 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=5d18db22ee924d578a2f7079502940eb, user.id=7a379f0e1f1f4c9c884081f682310ad9) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:18.202978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25da273-2c50-44ba-b816-b8295661755d None admin] RBAC: Adding query filter params (role.id=1d1b5b96118040f9a4fd019bbcd93071, scope.project.id=5d18db22ee924d578a2f7079502940eb, user.id=2a5cf324b6904a499ca6308498e53335) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:18.203296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25da273-2c50-44ba-b816-b8295661755d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:18.203640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99bcd0-8733-47bf-b8ab-b36887161754 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:18.204759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25da273-2c50-44ba-b816-b8295661755d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:18.205389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99bcd0-8733-47bf-b8ab-b36887161754 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:18.211758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 49/94] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:18 2022] GET /identity/v3/role_assignments?user.id=7a379f0e1f1f4c9c884081f682310ad9&scope.project.id=5d18db22ee924d578a2f7079502940eb&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:18.212862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99bcd0-8733-47bf-b8ab-b36887161754 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:18.227652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:18.235669 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 46/95] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:17 2022] GET /identity/v3/role_assignments?user.id=2a5cf324b6904a499ca6308498e53335&scope.project.id=5d18db22ee924d578a2f7079502940eb&role.id=1d1b5b96118040f9a4fd019bbcd93071 => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:18.380254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99bcd0-8733-47bf-b8ab-b36887161754 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:18.438206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a3888-20a6-47e4-a306-37c813941475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:18.439116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a3888-20a6-47e4-a306-37c813941475 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:18.439535 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a3888-20a6-47e4-a306-37c813941475 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:18.439972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a3888-20a6-47e4-a306-37c813941475 None admin] PATH_INFO: `/v3/projects/11e833036a6d48439a0b123384a4ff79` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:18.441702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a3888-20a6-47e4-a306-37c813941475 None admin] RBAC: Authorizing `identity:get_project(project_id=11e833036a6d48439a0b123384a4ff79)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:18.443397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a3888-20a6-47e4-a306-37c813941475 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:18.445223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 50/96] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:18 2022] GET /identity/v3/projects/11e833036a6d48439a0b123384a4ff79 => generated 312 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:18.451634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25da273-2c50-44ba-b816-b8295661755d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:18.595270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:18.605316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25da273-2c50-44ba-b816-b8295661755d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:18.642523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac9e96-abee-4862-a7a4-be71b7dcf6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:18.643398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac9e96-abee-4862-a7a4-be71b7dcf6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:18.643837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac9e96-abee-4862-a7a4-be71b7dcf6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:18.644244 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac9e96-abee-4862-a7a4-be71b7dcf6e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:18.645142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac9e96-abee-4862-a7a4-be71b7dcf6e2 None admin] RBAC: Adding query filter params (role.id=08a7c861d84b43d7a4da431f31f3c2d7, scope.project.id=11e833036a6d48439a0b123384a4ff79, user.id=2a5cf324b6904a499ca6308498e53335) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:18.645630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac9e96-abee-4862-a7a4-be71b7dcf6e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:18.647305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac9e96-abee-4862-a7a4-be71b7dcf6e2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:18.653826 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 47/97] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:18 2022] GET /identity/v3/role_assignments?user.id=2a5cf324b6904a499ca6308498e53335&scope.project.id=11e833036a6d48439a0b123384a4ff79&role.id=08a7c861d84b43d7a4da431f31f3c2d7 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:19.335254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fca9fb-3c2b-4b92-8fe8-861413eeda76 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:19.335756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fca9fb-3c2b-4b92-8fe8-861413eeda76 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:19.335826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fca9fb-3c2b-4b92-8fe8-861413eeda76 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:19.337298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 51/98] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:19.343215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6fbb5-dc4c-473f-a4cc-8e87beb26df0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:19.343663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6fbb5-dc4c-473f-a4cc-8e87beb26df0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:19.343957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6fbb5-dc4c-473f-a4cc-8e87beb26df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:19.362930 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f5c6fbb5-dc4c-473f-a4cc-8e87beb26df0 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:19.364132 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f5c6fbb5-dc4c-473f-a4cc-8e87beb26df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:19.440018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 48/99] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:19.476633 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fca9fb-3c2b-4b92-8fe8-861413eeda76 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:19.580349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:19.616257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fca9fb-3c2b-4b92-8fe8-861413eeda76 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:19.664131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05df4d-b796-426f-8964-f67d3a176d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:19.664772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05df4d-b796-426f-8964-f67d3a176d89 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:19.665049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05df4d-b796-426f-8964-f67d3a176d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:19.665369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05df4d-b796-426f-8964-f67d3a176d89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:19.679647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05df4d-b796-426f-8964-f67d3a176d89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-6c05df4d-b796-426f-8964-f67d3a176d89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:19.684480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:19.685315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:19.686088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 52/100] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:19.693321 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6fbb5-dc4c-473f-a4cc-8e87beb26df0 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:19.728527 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5512c23-bacd-4b4f-aab7-43c50a9e926d None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:19.728865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5512c23-bacd-4b4f-aab7-43c50a9e926d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:19.729102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5512c23-bacd-4b4f-aab7-43c50a9e926d None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:19.731157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 53/101] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:29:19.737383 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92218c-0e97-4776-87f5-60b499e973ec None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:19.737698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92218c-0e97-4776-87f5-60b499e973ec None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:19.737975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92218c-0e97-4776-87f5-60b499e973ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:19.774461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4a92218c-0e97-4776-87f5-60b499e973ec None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:19.776417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4a92218c-0e97-4776-87f5-60b499e973ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:19.865726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 54/102] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:19.906759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:19.907691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92218c-0e97-4776-87f5-60b499e973ec None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:19.943421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6fbb5-dc4c-473f-a4cc-8e87beb26df0 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:20.019641 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6a361-fea2-4d09-b48f-3b35c47bdd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:20.020777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6a361-fea2-4d09-b48f-3b35c47bdd30 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:20.021254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6a361-fea2-4d09-b48f-3b35c47bdd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:20.021784 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6a361-fea2-4d09-b48f-3b35c47bdd30 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:20.022622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6a361-fea2-4d09-b48f-3b35c47bdd30 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:20.023086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6a361-fea2-4d09-b48f-3b35c47bdd30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:20.033695 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6a361-fea2-4d09-b48f-3b35c47bdd30 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:20.043929 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 49/103] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:20.049503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6a361-fea2-4d09-b48f-3b35c47bdd30 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:20.068658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:20.144873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92218c-0e97-4776-87f5-60b499e973ec None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:20.225356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d46b8e-d7bf-48fe-8994-0917b9da7a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:20.226023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d46b8e-d7bf-48fe-8994-0917b9da7a79 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:20.227384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d46b8e-d7bf-48fe-8994-0917b9da7a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:20.227668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d46b8e-d7bf-48fe-8994-0917b9da7a79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:20.228947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d46b8e-d7bf-48fe-8994-0917b9da7a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:20.230427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d46b8e-d7bf-48fe-8994-0917b9da7a79 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:20.232009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 55/104] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:20.240582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d46b8e-d7bf-48fe-8994-0917b9da7a79 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:20.286170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6a361-fea2-4d09-b48f-3b35c47bdd30 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:20.354270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47dbf5-21f5-48ae-af21-bbbec0708d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:20.354926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47dbf5-21f5-48ae-af21-bbbec0708d47 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:20.355171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47dbf5-21f5-48ae-af21-bbbec0708d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:20.355439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47dbf5-21f5-48ae-af21-bbbec0708d47 None admin] PATH_INFO: `/v3/users/450d6ceea9a743bdb53b50979b70d5c1` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:20.387215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47dbf5-21f5-48ae-af21-bbbec0708d47 None admin] RBAC: Authorizing `identity:get_user(user_id=450d6ceea9a743bdb53b50979b70d5c1)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:20.388697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47dbf5-21f5-48ae-af21-bbbec0708d47 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:20.390715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 50/105] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:20 2022] GET /identity/v3/users/450d6ceea9a743bdb53b50979b70d5c1 => generated 289 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:20.398152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47dbf5-21f5-48ae-af21-bbbec0708d47 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:20.413874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:20.435745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d46b8e-d7bf-48fe-8994-0917b9da7a79 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:20.507337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:20.517956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f17dcc-9175-4d8d-bf34-893be87d89da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:20.518590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17dcc-9175-4d8d-bf34-893be87d89da None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:20.518872 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17dcc-9175-4d8d-bf34-893be87d89da None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:20.519373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17dcc-9175-4d8d-bf34-893be87d89da None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:20.520042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f17dcc-9175-4d8d-bf34-893be87d89da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:20.524043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f17dcc-9175-4d8d-bf34-893be87d89da None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:20.585210 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47dbf5-21f5-48ae-af21-bbbec0708d47 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:20.626043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d646d55-cc43-47ca-bb0f-3712ee69dc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:20.626662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d646d55-cc43-47ca-bb0f-3712ee69dc63 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:20.626999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d646d55-cc43-47ca-bb0f-3712ee69dc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:20.627273 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d646d55-cc43-47ca-bb0f-3712ee69dc63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:20.630564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 56/106] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:29:20 2022] POST /identity/v3/projects => generated 302 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:29:20.730469 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d646d55-cc43-47ca-bb0f-3712ee69dc63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:20.731865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d646d55-cc43-47ca-bb0f-3712ee69dc63 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:20.746276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:20.852054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 51/107] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:20.858666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f17dcc-9175-4d8d-bf34-893be87d89da None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:20.971725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1456042-3b81-4ae9-9c6e-8cf5d387fd9e None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:20.972110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1456042-3b81-4ae9-9c6e-8cf5d387fd9e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:20.972398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1456042-3b81-4ae9-9c6e-8cf5d387fd9e None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:20.973896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 52/108] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:20.979849 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57a27c-c138-45b8-b43e-0d2764081674 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:20.980127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57a27c-c138-45b8-b43e-0d2764081674 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:20.980415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57a27c-c138-45b8-b43e-0d2764081674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:20.996956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f17dcc-9175-4d8d-bf34-893be87d89da None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:21.019456 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-de57a27c-c138-45b8-b43e-0d2764081674 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:21.020691 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-de57a27c-c138-45b8-b43e-0d2764081674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:21.033731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3e064-d12c-46ec-a000-a38f07709303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:21.034852 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3e064-d12c-46ec-a000-a38f07709303 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:21.035335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3e064-d12c-46ec-a000-a38f07709303 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:21.035771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3e064-d12c-46ec-a000-a38f07709303 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:21.045814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3e064-d12c-46ec-a000-a38f07709303 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:21.045814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3e064-d12c-46ec-a000-a38f07709303 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:21.045814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 57/109] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:20 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:21.047157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3e064-d12c-46ec-a000-a38f07709303 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:21.133032 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 53/110] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:21.146093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:21.167713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57a27c-c138-45b8-b43e-0d2764081674 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:21.192049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3e064-d12c-46ec-a000-a38f07709303 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:21.222016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:21.231109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5c397-2e54-4a0b-9f6c-613ecb7ea779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:21.232054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5c397-2e54-4a0b-9f6c-613ecb7ea779 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:21.232595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5c397-2e54-4a0b-9f6c-613ecb7ea779 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:21.233025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5c397-2e54-4a0b-9f6c-613ecb7ea779 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:21.234010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5c397-2e54-4a0b-9f6c-613ecb7ea779 None admin] RBAC: Adding query filter params (role.id=1b625e8349d04de08806337ee4bd9478, scope.project.id=5d18db22ee924d578a2f7079502940eb, user.id=450d6ceea9a743bdb53b50979b70d5c1) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:21.234483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5c397-2e54-4a0b-9f6c-613ecb7ea779 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:21.236118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5c397-2e54-4a0b-9f6c-613ecb7ea779 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:21.242602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 58/111] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:21 2022] GET /identity/v3/role_assignments?user.id=450d6ceea9a743bdb53b50979b70d5c1&scope.project.id=5d18db22ee924d578a2f7079502940eb&role.id=1b625e8349d04de08806337ee4bd9478 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:21.260827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfa0fc-db80-4904-9c18-29a720347790 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:21.264154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfa0fc-db80-4904-9c18-29a720347790 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:21.264712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfa0fc-db80-4904-9c18-29a720347790 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:21.266527 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 59/112] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:29:21.279488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8622a35-80fd-4baa-9020-78c08773597a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:21.280121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8622a35-80fd-4baa-9020-78c08773597a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:21.280465 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8622a35-80fd-4baa-9020-78c08773597a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:21.313888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d8622a35-80fd-4baa-9020-78c08773597a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:21.315679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d8622a35-80fd-4baa-9020-78c08773597a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:21.337186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57a27c-c138-45b8-b43e-0d2764081674 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:21.421803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fe41f-6761-4e97-af09-614e109835f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:21.422515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fe41f-6761-4e97-af09-614e109835f6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:21.422749 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fe41f-6761-4e97-af09-614e109835f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:21.423024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fe41f-6761-4e97-af09-614e109835f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:21.459041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fe41f-6761-4e97-af09-614e109835f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-748fe41f-6761-4e97-af09-614e109835f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:21.464442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:21.465439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:21.466378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:21.466378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:21.466378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:21.466378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:21.466378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:21.466378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:21.466378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:21.466378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:21.466378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:21.466378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:21.466378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:21.466378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:21.467143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 54/113] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:21.476048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 60/114] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:21.481263 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fe41f-6761-4e97-af09-614e109835f6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:21.501207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8622a35-80fd-4baa-9020-78c08773597a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:21.601256 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:21.660305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8622a35-80fd-4baa-9020-78c08773597a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:21.697765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef584428-3a0e-4e45-b789-1c99b4bf6e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:21.698520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef584428-3a0e-4e45-b789-1c99b4bf6e4f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:21.698805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef584428-3a0e-4e45-b789-1c99b4bf6e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:21.699079 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef584428-3a0e-4e45-b789-1c99b4bf6e4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:21.704534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:21.717822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fe41f-6761-4e97-af09-614e109835f6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:21.721121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef584428-3a0e-4e45-b789-1c99b4bf6e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-ef584428-3a0e-4e45-b789-1c99b4bf6e4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:21.726442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:21.727275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:21.728040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:21.728040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:21.728040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:21.728040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:21.728040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:21.728040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:21.728040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:21.728040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:21.728040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:21.728040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:21.728040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:21.728040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:21.733071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 61/115] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:29:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:21.736123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8856b9-e2cc-496d-a382-3435a8a5d881 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:21.737607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8856b9-e2cc-496d-a382-3435a8a5d881 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:21.737892 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8856b9-e2cc-496d-a382-3435a8a5d881 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:21.740235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 62/116] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:21.740797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8856b9-e2cc-496d-a382-3435a8a5d881 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:21.779038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad99d7-e302-48c6-9d15-12f97f5154a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:21.780077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad99d7-e302-48c6-9d15-12f97f5154a7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:21.780366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad99d7-e302-48c6-9d15-12f97f5154a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:21.780912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad99d7-e302-48c6-9d15-12f97f5154a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:21.781613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad99d7-e302-48c6-9d15-12f97f5154a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:21.782178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad99d7-e302-48c6-9d15-12f97f5154a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:21.784291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad99d7-e302-48c6-9d15-12f97f5154a7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:21.796421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 55/117] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:21.797703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102555a-ee57-4c4b-a606-8c1144996835 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:21.798565 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102555a-ee57-4c4b-a606-8c1144996835 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:21.798851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102555a-ee57-4c4b-a606-8c1144996835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:21.823603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a102555a-ee57-4c4b-a606-8c1144996835 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:21.825610 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a102555a-ee57-4c4b-a606-8c1144996835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:21.870636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8856b9-e2cc-496d-a382-3435a8a5d881 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:21.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:21.907683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 56/118] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:21.909478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355898dd-3d4d-407a-a0e4-899f437e4abc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:21.910009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355898dd-3d4d-407a-a0e4-899f437e4abc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:21.910459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355898dd-3d4d-407a-a0e4-899f437e4abc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:21.912405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 57/119] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:21.913301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355898dd-3d4d-407a-a0e4-899f437e4abc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:21.920477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef82ab-8a57-4f1c-839a-3823a9239bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:21.921158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef82ab-8a57-4f1c-839a-3823a9239bcf None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:21.921442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef82ab-8a57-4f1c-839a-3823a9239bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:21.921768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef82ab-8a57-4f1c-839a-3823a9239bcf None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:21.922503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef82ab-8a57-4f1c-839a-3823a9239bcf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:21.922807 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef82ab-8a57-4f1c-839a-3823a9239bcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:21.924607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef82ab-8a57-4f1c-839a-3823a9239bcf None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:21.938044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 63/120] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:29:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:21.943060 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa7880-255f-41ed-acc7-96f1355ef2ea None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:21.943349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa7880-255f-41ed-acc7-96f1355ef2ea None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:21.943627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa7880-255f-41ed-acc7-96f1355ef2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:21.968954 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-dfaa7880-255f-41ed-acc7-96f1355ef2ea None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:21.970180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dfaa7880-255f-41ed-acc7-96f1355ef2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:22.031048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:22.045798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355898dd-3d4d-407a-a0e4-899f437e4abc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:22.051639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 64/121] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:22.051639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa7880-255f-41ed-acc7-96f1355ef2ea None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:22.107197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad7295-399b-4773-97c8-9c6f3736e866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:22.107857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad7295-399b-4773-97c8-9c6f3736e866 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:22.108192 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad7295-399b-4773-97c8-9c6f3736e866 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:22.108498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad7295-399b-4773-97c8-9c6f3736e866 None admin] PATH_INFO: `/v3/users/2a5cf324b6904a499ca6308498e53335` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:22.109759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad7295-399b-4773-97c8-9c6f3736e866 None admin] RBAC: Authorizing `identity:get_user(user_id=2a5cf324b6904a499ca6308498e53335)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:22.111242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad7295-399b-4773-97c8-9c6f3736e866 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:22.113181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 58/122] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:21 2022] GET /identity/v3/users/2a5cf324b6904a499ca6308498e53335 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:22.117882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad7295-399b-4773-97c8-9c6f3736e866 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:22.210833 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:22.215203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa7880-255f-41ed-acc7-96f1355ef2ea None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:22.252572 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96120380-2b9c-43ea-b7a7-3d91e83c37a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:22.253218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96120380-2b9c-43ea-b7a7-3d91e83c37a6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:22.253613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96120380-2b9c-43ea-b7a7-3d91e83c37a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:22.253885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96120380-2b9c-43ea-b7a7-3d91e83c37a6 None admin] PATH_INFO: `/v3/users/2a5cf324b6904a499ca6308498e53335` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:22.255049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96120380-2b9c-43ea-b7a7-3d91e83c37a6 None admin] RBAC: Authorizing `identity:get_user(user_id=2a5cf324b6904a499ca6308498e53335)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:22.256487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96120380-2b9c-43ea-b7a7-3d91e83c37a6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:22.258108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 65/123] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:22 2022] GET /identity/v3/users/2a5cf324b6904a499ca6308498e53335 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:22.258211 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad7295-399b-4773-97c8-9c6f3736e866 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:22.264640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96120380-2b9c-43ea-b7a7-3d91e83c37a6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:22.297343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e1548-c9b9-4619-aef1-45fa41825816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:22.298473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e1548-c9b9-4619-aef1-45fa41825816 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:22.298947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e1548-c9b9-4619-aef1-45fa41825816 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:22.299547 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e1548-c9b9-4619-aef1-45fa41825816 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:22.311795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e1548-c9b9-4619-aef1-45fa41825816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-296e1548-c9b9-4619-aef1-45fa41825816 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:22.317178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:22.318027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:22.318783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:22.318783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:22.318783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:22.318783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:22.318783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:22.318783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:22.318783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:22.318783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:22.318783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:22.318783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:22.318783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:22.318783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:22.321022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 59/124] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:29:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:22.322356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e1548-c9b9-4619-aef1-45fa41825816 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:22.343113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:22.376455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96120380-2b9c-43ea-b7a7-3d91e83c37a6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:22.413681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1724f-a8cd-4708-abad-579928a0c741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:22.414325 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1724f-a8cd-4708-abad-579928a0c741 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:22.414669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1724f-a8cd-4708-abad-579928a0c741 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:22.414935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1724f-a8cd-4708-abad-579928a0c741 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:22.426487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1724f-a8cd-4708-abad-579928a0c741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b1c1724f-a8cd-4708-abad-579928a0c741 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:22.432247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:22.433072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:22.433863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 66/125] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:22.434563 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1724f-a8cd-4708-abad-579928a0c741 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:22.446268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e1548-c9b9-4619-aef1-45fa41825816 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:22.479038 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:22.499082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8c164-a0d2-4f1d-9b0f-c2ceacd90433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:22.500000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8c164-a0d2-4f1d-9b0f-c2ceacd90433 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:22.500369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8c164-a0d2-4f1d-9b0f-c2ceacd90433 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:22.500715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8c164-a0d2-4f1d-9b0f-c2ceacd90433 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:22.511258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8c164-a0d2-4f1d-9b0f-c2ceacd90433 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:22.513212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8c164-a0d2-4f1d-9b0f-c2ceacd90433 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:22.516110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 60/126] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:22 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:22.517054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8c164-a0d2-4f1d-9b0f-c2ceacd90433 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:22.563563 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1724f-a8cd-4708-abad-579928a0c741 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:22.611399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44961b9-c8f0-475f-8fab-b675f90a86b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:22.612402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44961b9-c8f0-475f-8fab-b675f90a86b4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:22.612907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44961b9-c8f0-475f-8fab-b675f90a86b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:22.613377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44961b9-c8f0-475f-8fab-b675f90a86b4 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:22.614740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44961b9-c8f0-475f-8fab-b675f90a86b4 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:22.616399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44961b9-c8f0-475f-8fab-b675f90a86b4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:22.618356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 67/127] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:22 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:22.619924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44961b9-c8f0-475f-8fab-b675f90a86b4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:22.644951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:22.667999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8c164-a0d2-4f1d-9b0f-c2ceacd90433 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:22.715370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a9c26-063b-4dfb-8ee1-24abd7075adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:22.716490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9c26-063b-4dfb-8ee1-24abd7075adb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:22.716782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9c26-063b-4dfb-8ee1-24abd7075adb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:22.717061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9c26-063b-4dfb-8ee1-24abd7075adb None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:22.717896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a9c26-063b-4dfb-8ee1-24abd7075adb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:22.718345 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a9c26-063b-4dfb-8ee1-24abd7075adb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:22.719992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a9c26-063b-4dfb-8ee1-24abd7075adb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:22.730348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:22.733116 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 61/128] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:22.738149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a9c26-063b-4dfb-8ee1-24abd7075adb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:22.742401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44961b9-c8f0-475f-8fab-b675f90a86b4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:22.780077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec75fa-edb7-4c05-980d-b9853fd50c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:22.781199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec75fa-edb7-4c05-980d-b9853fd50c97 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:22.781717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec75fa-edb7-4c05-980d-b9853fd50c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:22.782198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec75fa-edb7-4c05-980d-b9853fd50c97 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:22.783118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec75fa-edb7-4c05-980d-b9853fd50c97 None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:22.783640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec75fa-edb7-4c05-980d-b9853fd50c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:22.785314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec75fa-edb7-4c05-980d-b9853fd50c97 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:22.795288 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 68/129] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:22.796474 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec75fa-edb7-4c05-980d-b9853fd50c97 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:22.859653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a9c26-063b-4dfb-8ee1-24abd7075adb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:22.898478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec016e-c894-4230-9f29-5a35d1377c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:22.899618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec016e-c894-4230-9f29-5a35d1377c40 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:22.900097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec016e-c894-4230-9f29-5a35d1377c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:22.901115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec016e-c894-4230-9f29-5a35d1377c40 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb/users/2a5cf324b6904a499ca6308498e53335/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:22.903343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec016e-c894-4230-9f29-5a35d1377c40 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d18db22ee924d578a2f7079502940eb, user_id=2a5cf324b6904a499ca6308498e53335, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:22.905099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec016e-c894-4230-9f29-5a35d1377c40 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:22.927276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec75fa-edb7-4c05-980d-b9853fd50c97 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:22.965313 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca814273-c080-448a-8fc8-3965de09efc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:22.966006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca814273-c080-448a-8fc8-3965de09efc7 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:22.966289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca814273-c080-448a-8fc8-3965de09efc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:22.966564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca814273-c080-448a-8fc8-3965de09efc7 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb/users/2a5cf324b6904a499ca6308498e53335/roles/1d1b5b96118040f9a4fd019bbcd93071` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:22.968831 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca814273-c080-448a-8fc8-3965de09efc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d18db22ee924d578a2f7079502940eb, user_id=2a5cf324b6904a499ca6308498e53335, role_id=1d1b5b96118040f9a4fd019bbcd93071)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:22.970350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca814273-c080-448a-8fc8-3965de09efc7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:22.991888 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 62/130] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:29:22 2022] PUT /identity/v3/projects/5d18db22ee924d578a2f7079502940eb/users/2a5cf324b6904a499ca6308498e53335/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:29:22.993062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec016e-c894-4230-9f29-5a35d1377c40 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:23.008740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:23.069110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 69/131] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:29:22 2022] PUT /identity/v3/projects/5d18db22ee924d578a2f7079502940eb/users/2a5cf324b6904a499ca6308498e53335/roles/1d1b5b96118040f9a4fd019bbcd93071 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:29:23.074756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca814273-c080-448a-8fc8-3965de09efc7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:23.126443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:23.226516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec016e-c894-4230-9f29-5a35d1377c40 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:23.289933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca814273-c080-448a-8fc8-3965de09efc7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:23.300976 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bc8a5-1e97-4f24-961b-f1d2a20cf556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:23.302012 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bc8a5-1e97-4f24-961b-f1d2a20cf556 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:23.302503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bc8a5-1e97-4f24-961b-f1d2a20cf556 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:23.302981 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bc8a5-1e97-4f24-961b-f1d2a20cf556 None admin] PATH_INFO: `/v3/users/7a379f0e1f1f4c9c884081f682310ad9` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:23.304717 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5bc8a5-1e97-4f24-961b-f1d2a20cf556 None admin] RBAC: Authorizing `identity:get_user(user_id=7a379f0e1f1f4c9c884081f682310ad9)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:23.306451 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5bc8a5-1e97-4f24-961b-f1d2a20cf556 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:23.308724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 63/132] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:22 2022] GET /identity/v3/users/7a379f0e1f1f4c9c884081f682310ad9 => generated 247 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:23.316247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bc8a5-1e97-4f24-961b-f1d2a20cf556 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:23.339106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6be77-bed5-46ba-bd12-d79042881102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:23.339106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6be77-bed5-46ba-bd12-d79042881102 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:23.339407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6be77-bed5-46ba-bd12-d79042881102 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:23.339624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6be77-bed5-46ba-bd12-d79042881102 None admin] PATH_INFO: `/v3/users/2a5cf324b6904a499ca6308498e53335` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:23.340784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6be77-bed5-46ba-bd12-d79042881102 None admin] RBAC: Authorizing `identity:get_user(user_id=2a5cf324b6904a499ca6308498e53335)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:23.342981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6be77-bed5-46ba-bd12-d79042881102 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:23.344363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 70/133] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:23 2022] GET /identity/v3/users/2a5cf324b6904a499ca6308498e53335 => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:23.349582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6be77-bed5-46ba-bd12-d79042881102 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:23.420799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:23.446188 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:23.447598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bc8a5-1e97-4f24-961b-f1d2a20cf556 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:23.464384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6be77-bed5-46ba-bd12-d79042881102 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:23.483912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf174920-45e9-43e3-bb7a-09aaacb50b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:23.484554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf174920-45e9-43e3-bb7a-09aaacb50b95 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:23.484839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf174920-45e9-43e3-bb7a-09aaacb50b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:23.485116 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf174920-45e9-43e3-bb7a-09aaacb50b95 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:23.486328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf174920-45e9-43e3-bb7a-09aaacb50b95 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:23.487798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf174920-45e9-43e3-bb7a-09aaacb50b95 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:23.489464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 64/134] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:23 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:23.495902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf174920-45e9-43e3-bb7a-09aaacb50b95 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:23.504585 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f641de-a138-425e-8164-4231e9334e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:23.505240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f641de-a138-425e-8164-4231e9334e9f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:23.505540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f641de-a138-425e-8164-4231e9334e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:23.505950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f641de-a138-425e-8164-4231e9334e9f None admin] PATH_INFO: `/v3/projects/11e833036a6d48439a0b123384a4ff79` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:23.507133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f641de-a138-425e-8164-4231e9334e9f None admin] RBAC: Authorizing `identity:get_project(project_id=11e833036a6d48439a0b123384a4ff79)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:23.508783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f641de-a138-425e-8164-4231e9334e9f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:23.510648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 71/135] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:23 2022] GET /identity/v3/projects/11e833036a6d48439a0b123384a4ff79 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:23.516587 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f641de-a138-425e-8164-4231e9334e9f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:23.615855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf174920-45e9-43e3-bb7a-09aaacb50b95 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:23.634286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f641de-a138-425e-8164-4231e9334e9f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:23.656798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccc7f8-9539-4fec-971d-8576f81b9a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:23.658041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccc7f8-9539-4fec-971d-8576f81b9a84 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:23.658335 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccc7f8-9539-4fec-971d-8576f81b9a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:23.658621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccc7f8-9539-4fec-971d-8576f81b9a84 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb/users/7a379f0e1f1f4c9c884081f682310ad9/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:23.661170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccc7f8-9539-4fec-971d-8576f81b9a84 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d18db22ee924d578a2f7079502940eb, user_id=7a379f0e1f1f4c9c884081f682310ad9, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:23.662868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccc7f8-9539-4fec-971d-8576f81b9a84 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:23.683792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526a274-014c-49fc-9db5-c3286dd3f2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:23.684971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526a274-014c-49fc-9db5-c3286dd3f2c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:23.685451 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526a274-014c-49fc-9db5-c3286dd3f2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:23.685956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526a274-014c-49fc-9db5-c3286dd3f2c6 None admin] PATH_INFO: `/v3/projects/11e833036a6d48439a0b123384a4ff79/users/2a5cf324b6904a499ca6308498e53335/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:23.688435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526a274-014c-49fc-9db5-c3286dd3f2c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=11e833036a6d48439a0b123384a4ff79, user_id=2a5cf324b6904a499ca6308498e53335, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:23.690063 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526a274-014c-49fc-9db5-c3286dd3f2c6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:23.759488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 65/136] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:29:23 2022] PUT /identity/v3/projects/5d18db22ee924d578a2f7079502940eb/users/7a379f0e1f1f4c9c884081f682310ad9/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:29:23.761562 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace9564-5636-4f69-8947-476ab743855e None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:23.761562 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace9564-5636-4f69-8947-476ab743855e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:23.761562 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace9564-5636-4f69-8947-476ab743855e None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:23.762848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 66/137] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:23.780647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0458a8-6c66-4681-8f22-c35f1a162adc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:23.780965 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0458a8-6c66-4681-8f22-c35f1a162adc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:23.781205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0458a8-6c66-4681-8f22-c35f1a162adc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:23.787840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 72/138] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:29:23 2022] PUT /identity/v3/projects/11e833036a6d48439a0b123384a4ff79/users/2a5cf324b6904a499ca6308498e53335/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:29:23.794993 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 67/139] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:23 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:23.795582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc64954-e8a7-4bb7-894d-bc90fea1a12c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:23.795907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc64954-e8a7-4bb7-894d-bc90fea1a12c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:23.796247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc64954-e8a7-4bb7-894d-bc90fea1a12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:23.808206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e74f5-3401-4c6d-9c88-a96983964fce None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:23.808507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e74f5-3401-4c6d-9c88-a96983964fce None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:23.808771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e74f5-3401-4c6d-9c88-a96983964fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:23.821235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:23.843875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1a8e74f5-3401-4c6d-9c88-a96983964fce None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:23.845166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1a8e74f5-3401-4c6d-9c88-a96983964fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:23.846954 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-fbc64954-e8a7-4bb7-894d-bc90fea1a12c None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:23.848275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fbc64954-e8a7-4bb7-894d-bc90fea1a12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:23.867268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:23.996541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 73/140] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:23.998891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 68/141] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 09 18:29:24.022845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc64954-e8a7-4bb7-894d-bc90fea1a12c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:24.025472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e74f5-3401-4c6d-9c88-a96983964fce None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:24.136211 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e74f5-3401-4c6d-9c88-a96983964fce None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:24.137148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc64954-e8a7-4bb7-894d-bc90fea1a12c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:24.170205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:24.177802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228cdd5-dd4d-470c-ae89-f0b1e6a89ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:24.177977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228cdd5-dd4d-470c-ae89-f0b1e6a89ede None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:24.178206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228cdd5-dd4d-470c-ae89-f0b1e6a89ede None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:24.178395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228cdd5-dd4d-470c-ae89-f0b1e6a89ede None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:24.180613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228cdd5-dd4d-470c-ae89-f0b1e6a89ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:24.181450 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a163d9-9abd-430f-b04d-3c7a02356d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:24.185613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228cdd5-dd4d-470c-ae89-f0b1e6a89ede None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:24.185750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a163d9-9abd-430f-b04d-3c7a02356d86 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:24.185750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a163d9-9abd-430f-b04d-3c7a02356d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:24.185750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a163d9-9abd-430f-b04d-3c7a02356d86 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:24.185961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 69/142] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:24.191779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228cdd5-dd4d-470c-ae89-f0b1e6a89ede None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:24.196840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a163d9-9abd-430f-b04d-3c7a02356d86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-d8a163d9-9abd-430f-b04d-3c7a02356d86 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:24.202686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:24.203801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:24.204592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 74/143] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:24.210290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a163d9-9abd-430f-b04d-3c7a02356d86 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:24.217870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:24.328064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a163d9-9abd-430f-b04d-3c7a02356d86 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:24.333072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228cdd5-dd4d-470c-ae89-f0b1e6a89ede None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:24.367361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf921baa-d141-44fe-9c1c-000d5ae4f5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:24.368086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf921baa-d141-44fe-9c1c-000d5ae4f5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:24.368436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf921baa-d141-44fe-9c1c-000d5ae4f5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:24.368777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf921baa-d141-44fe-9c1c-000d5ae4f5a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:24.370113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf921baa-d141-44fe-9c1c-000d5ae4f5a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:24.370580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf921baa-d141-44fe-9c1c-000d5ae4f5a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:24.372748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf921baa-d141-44fe-9c1c-000d5ae4f5a0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:24.382398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 75/144] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:24.389736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf921baa-d141-44fe-9c1c-000d5ae4f5a0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:24.470269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0a7b4-bd13-481e-9f8c-63264cfbb868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:24.470904 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0a7b4-bd13-481e-9f8c-63264cfbb868 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:24.471188 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0a7b4-bd13-481e-9f8c-63264cfbb868 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:24.471468 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0a7b4-bd13-481e-9f8c-63264cfbb868 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:24.472149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0a7b4-bd13-481e-9f8c-63264cfbb868 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:24.473633 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0a7b4-bd13-481e-9f8c-63264cfbb868 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:24.493710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:24.519380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf921baa-d141-44fe-9c1c-000d5ae4f5a0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:24.554754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcc7da-646d-42ad-be64-71f20efcb351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:24.555624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcc7da-646d-42ad-be64-71f20efcb351 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:24.556052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcc7da-646d-42ad-be64-71f20efcb351 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:24.556464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcc7da-646d-42ad-be64-71f20efcb351 None admin] PATH_INFO: `/v3/users/450d6ceea9a743bdb53b50979b70d5c1` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:24.558631 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcc7da-646d-42ad-be64-71f20efcb351 None admin] RBAC: Authorizing `identity:get_user(user_id=450d6ceea9a743bdb53b50979b70d5c1)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:24.560250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcc7da-646d-42ad-be64-71f20efcb351 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:24.562144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 76/145] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:24 2022] GET /identity/v3/users/450d6ceea9a743bdb53b50979b70d5c1 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:24.567788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcc7da-646d-42ad-be64-71f20efcb351 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:24.610718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 70/146] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:29:24 2022] POST /identity/v3/users => generated 283 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:29:24.689650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcc7da-646d-42ad-be64-71f20efcb351 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:24.722771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:24.728169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c04e8d-eaeb-41d7-a901-2af570b85056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:24.728900 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c04e8d-eaeb-41d7-a901-2af570b85056 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:24.729186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c04e8d-eaeb-41d7-a901-2af570b85056 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:24.729467 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c04e8d-eaeb-41d7-a901-2af570b85056 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:24.730903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c04e8d-eaeb-41d7-a901-2af570b85056 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:24.732429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c04e8d-eaeb-41d7-a901-2af570b85056 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:24.734384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 77/147] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:24 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:24.740411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0a7b4-bd13-481e-9f8c-63264cfbb868 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:24.823478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:24.877173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0a7b4-bd13-481e-9f8c-63264cfbb868 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:24.921979 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd882f0e-f29c-46b0-bdb2-6920c88579bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:24.922655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd882f0e-f29c-46b0-bdb2-6920c88579bf None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:24.922942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd882f0e-f29c-46b0-bdb2-6920c88579bf None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:24.923221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd882f0e-f29c-46b0-bdb2-6920c88579bf None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb/users/450d6ceea9a743bdb53b50979b70d5c1/roles/1b625e8349d04de08806337ee4bd9478` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:24.925194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd882f0e-f29c-46b0-bdb2-6920c88579bf None admin] RBAC: Authorizing `identity:create_grant(project_id=5d18db22ee924d578a2f7079502940eb, user_id=450d6ceea9a743bdb53b50979b70d5c1, role_id=1b625e8349d04de08806337ee4bd9478)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:24.930762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd882f0e-f29c-46b0-bdb2-6920c88579bf None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:25.015686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 71/148] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:29:24 2022] PUT /identity/v3/projects/5d18db22ee924d578a2f7079502940eb/users/450d6ceea9a743bdb53b50979b70d5c1/roles/1b625e8349d04de08806337ee4bd9478 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:29:25.929832 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e653e-9558-479b-8df5-367ca68cdc63 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:25.930457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e653e-9558-479b-8df5-367ca68cdc63 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:25.930457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e653e-9558-479b-8df5-367ca68cdc63 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:25.931772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 78/149] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:25.938862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a11d3b-aeae-48c7-b6da-9156d41c787f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:25.939395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a11d3b-aeae-48c7-b6da-9156d41c787f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:25.939395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a11d3b-aeae-48c7-b6da-9156d41c787f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:25.959050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-39a11d3b-aeae-48c7-b6da-9156d41c787f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:25.960308 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-39a11d3b-aeae-48c7-b6da-9156d41c787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:26.049492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 72/150] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:26.086850 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e653e-9558-479b-8df5-367ca68cdc63 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:26.216093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e653e-9558-479b-8df5-367ca68cdc63 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:26.252757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb7721-b4dd-43f5-990b-0e28e8f0f19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:26.253395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7721-b4dd-43f5-990b-0e28e8f0f19e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:26.253712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7721-b4dd-43f5-990b-0e28e8f0f19e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:26.253981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7721-b4dd-43f5-990b-0e28e8f0f19e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:26.268396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb7721-b4dd-43f5-990b-0e28e8f0f19e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-c4cb7721-b4dd-43f5-990b-0e28e8f0f19e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:26.274118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:26.274946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 79/151] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:26.283711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a11d3b-aeae-48c7-b6da-9156d41c787f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:26.348800 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:26.393321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f640278-18c7-4a26-8446-62e5b8b83827 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:26.393723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f640278-18c7-4a26-8446-62e5b8b83827 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:26.394001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f640278-18c7-4a26-8446-62e5b8b83827 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:26.395369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 80/152] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:26.403521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93ddf5-ac0a-443f-9421-03f557b5aaee None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:26.403860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93ddf5-ac0a-443f-9421-03f557b5aaee None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:26.404134 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93ddf5-ac0a-443f-9421-03f557b5aaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:26.428607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cc93ddf5-ac0a-443f-9421-03f557b5aaee None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:26.430389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cc93ddf5-ac0a-443f-9421-03f557b5aaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:26.441881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a11d3b-aeae-48c7-b6da-9156d41c787f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:26.484750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceceffc7-44a6-419d-aea6-5aab2ffa33b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:26.485293 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceceffc7-44a6-419d-aea6-5aab2ffa33b5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:26.485601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceceffc7-44a6-419d-aea6-5aab2ffa33b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:26.485878 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceceffc7-44a6-419d-aea6-5aab2ffa33b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:26.486564 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceceffc7-44a6-419d-aea6-5aab2ffa33b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:26.486874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceceffc7-44a6-419d-aea6-5aab2ffa33b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:26.488294 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceceffc7-44a6-419d-aea6-5aab2ffa33b5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:26.508137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 73/153] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:26.514259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94d892-eda7-4a07-8b89-59ba4495c599 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:26.514663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94d892-eda7-4a07-8b89-59ba4495c599 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:26.515025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94d892-eda7-4a07-8b89-59ba4495c599 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:26.517001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 74/154] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:26.519533 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94d892-eda7-4a07-8b89-59ba4495c599 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:26.522468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:26.531947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 81/155] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:26.533320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05669a6-63b1-442a-aee5-e20c283175c9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:26.533635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05669a6-63b1-442a-aee5-e20c283175c9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:26.533912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05669a6-63b1-442a-aee5-e20c283175c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:26.553455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d05669a6-63b1-442a-aee5-e20c283175c9 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:26.554931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d05669a6-63b1-442a-aee5-e20c283175c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:26.628467 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 82/156] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:26.629407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05669a6-63b1-442a-aee5-e20c283175c9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:26.708822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94d892-eda7-4a07-8b89-59ba4495c599 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:26.721924 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:26.766693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924e2a5-75d7-4a10-8ce4-2fb049f1a5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:26.767248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924e2a5-75d7-4a10-8ce4-2fb049f1a5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:26.767636 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924e2a5-75d7-4a10-8ce4-2fb049f1a5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:26.767911 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924e2a5-75d7-4a10-8ce4-2fb049f1a5f0 None admin] PATH_INFO: `/v3/users/2a5cf324b6904a499ca6308498e53335` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:26.771244 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924e2a5-75d7-4a10-8ce4-2fb049f1a5f0 None admin] RBAC: Authorizing `identity:get_user(user_id=2a5cf324b6904a499ca6308498e53335)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:26.772715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924e2a5-75d7-4a10-8ce4-2fb049f1a5f0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:26.781642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05669a6-63b1-442a-aee5-e20c283175c9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:26.781816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 75/157] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:26 2022] GET /identity/v3/users/2a5cf324b6904a499ca6308498e53335 => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:26.781816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924e2a5-75d7-4a10-8ce4-2fb049f1a5f0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:26.813925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:26.833034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af305455-2c9c-407f-a2f3-6705053bd60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:26.833730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af305455-2c9c-407f-a2f3-6705053bd60d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:26.834016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af305455-2c9c-407f-a2f3-6705053bd60d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:26.834292 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af305455-2c9c-407f-a2f3-6705053bd60d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:26.856178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af305455-2c9c-407f-a2f3-6705053bd60d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-af305455-2c9c-407f-a2f3-6705053bd60d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:26.866730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:26.867714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:26.868489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 83/158] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:29:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:26.870187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af305455-2c9c-407f-a2f3-6705053bd60d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:26.996482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924e2a5-75d7-4a10-8ce4-2fb049f1a5f0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:27.071921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af305455-2c9c-407f-a2f3-6705053bd60d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:27.083592 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60117cea-34d4-456a-bdab-63ab69bf6d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:27.084239 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60117cea-34d4-456a-bdab-63ab69bf6d5b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:27.084522 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60117cea-34d4-456a-bdab-63ab69bf6d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:27.084788 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60117cea-34d4-456a-bdab-63ab69bf6d5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:27.118485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60117cea-34d4-456a-bdab-63ab69bf6d5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-60117cea-34d4-456a-bdab-63ab69bf6d5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:27.124466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:27.125328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:27.126181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 76/159] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:29:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:27.126834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60117cea-34d4-456a-bdab-63ab69bf6d5b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:27.143737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ba78b-bafd-4e97-a564-7fd8ac3aff2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:27.144761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ba78b-bafd-4e97-a564-7fd8ac3aff2d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:27.145263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ba78b-bafd-4e97-a564-7fd8ac3aff2d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:27.145769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ba78b-bafd-4e97-a564-7fd8ac3aff2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:27.218635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:27.249977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:27.266532 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ba78b-bafd-4e97-a564-7fd8ac3aff2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:27.268084 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ba78b-bafd-4e97-a564-7fd8ac3aff2d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:27.383984 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60117cea-34d4-456a-bdab-63ab69bf6d5b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:27.428075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 84/160] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:27.428933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ba78b-bafd-4e97-a564-7fd8ac3aff2d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:27.445894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce844b4-5530-48a9-bc6d-0b5a4e221be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:27.446881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce844b4-5530-48a9-bc6d-0b5a4e221be2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:27.447509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce844b4-5530-48a9-bc6d-0b5a4e221be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:27.448074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce844b4-5530-48a9-bc6d-0b5a4e221be2 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:27.448978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce844b4-5530-48a9-bc6d-0b5a4e221be2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:27.449583 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce844b4-5530-48a9-bc6d-0b5a4e221be2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:27.451177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce844b4-5530-48a9-bc6d-0b5a4e221be2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:27.469081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 77/161] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:29:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:27.470350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce844b4-5530-48a9-bc6d-0b5a4e221be2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:27.518617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:27.538903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ba78b-bafd-4e97-a564-7fd8ac3aff2d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:27.592427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8662a9c-6b32-4ac2-b83f-37cc587985ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:27.593478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8662a9c-6b32-4ac2-b83f-37cc587985ea None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:27.593946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8662a9c-6b32-4ac2-b83f-37cc587985ea None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:27.594379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8662a9c-6b32-4ac2-b83f-37cc587985ea None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:27.596993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8662a9c-6b32-4ac2-b83f-37cc587985ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:27.596993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8662a9c-6b32-4ac2-b83f-37cc587985ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:27.598607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8662a9c-6b32-4ac2-b83f-37cc587985ea None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:27.613895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 85/162] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:27.615132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8662a9c-6b32-4ac2-b83f-37cc587985ea None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:27.654533 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:27.663478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce844b4-5530-48a9-bc6d-0b5a4e221be2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:27.725964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05806c-c1bd-4f8d-b213-8479683eddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:27.727085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05806c-c1bd-4f8d-b213-8479683eddb9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:27.727579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05806c-c1bd-4f8d-b213-8479683eddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:27.728077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05806c-c1bd-4f8d-b213-8479683eddb9 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:27.729744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8662a9c-6b32-4ac2-b83f-37cc587985ea None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:27.730643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05806c-c1bd-4f8d-b213-8479683eddb9 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:27.732556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05806c-c1bd-4f8d-b213-8479683eddb9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:27.734499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 78/163] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:27 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:27.735492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05806c-c1bd-4f8d-b213-8479683eddb9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:27.766253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317d5c5-2409-4309-9f11-a5e5920757d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:27.767034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317d5c5-2409-4309-9f11-a5e5920757d7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:27.767319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317d5c5-2409-4309-9f11-a5e5920757d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:27.767593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317d5c5-2409-4309-9f11-a5e5920757d7 None admin] PATH_INFO: `/v3/users/2a5cf324b6904a499ca6308498e53335` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:27.768823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317d5c5-2409-4309-9f11-a5e5920757d7 None admin] RBAC: Authorizing `identity:get_user(user_id=2a5cf324b6904a499ca6308498e53335)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:27.770560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317d5c5-2409-4309-9f11-a5e5920757d7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:27.772855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 86/164] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:27 2022] GET /identity/v3/users/2a5cf324b6904a499ca6308498e53335 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:27.773878 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1290f-aef4-4d3c-8fd5-041038765a58 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:27.774436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1290f-aef4-4d3c-8fd5-041038765a58 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:27.774779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1290f-aef4-4d3c-8fd5-041038765a58 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:27.780714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 87/165] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:27.780714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1290f-aef4-4d3c-8fd5-041038765a58 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:27.793218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:27.902845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05806c-c1bd-4f8d-b213-8479683eddb9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:27.903788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1290f-aef4-4d3c-8fd5-041038765a58 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:27.948638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa70570-f4ff-4349-b4de-e400c2e5a277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:27.949464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09838fd7-43f3-4b4c-9631-dbb3395b38ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:27.949951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa70570-f4ff-4349-b4de-e400c2e5a277 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:27.950467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa70570-f4ff-4349-b4de-e400c2e5a277 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:27.950595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09838fd7-43f3-4b4c-9631-dbb3395b38ee None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:27.950992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09838fd7-43f3-4b4c-9631-dbb3395b38ee None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:27.951278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa70570-f4ff-4349-b4de-e400c2e5a277 None admin] PATH_INFO: `/v3/users/7a379f0e1f1f4c9c884081f682310ad9` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:27.951378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09838fd7-43f3-4b4c-9631-dbb3395b38ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:27.952061 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09838fd7-43f3-4b4c-9631-dbb3395b38ee None admin] RBAC: Adding query filter params (role.id=08a7c861d84b43d7a4da431f31f3c2d7, scope.project.id=5d18db22ee924d578a2f7079502940eb, user.id=2a5cf324b6904a499ca6308498e53335) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:27.952359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09838fd7-43f3-4b4c-9631-dbb3395b38ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:27.953114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa70570-f4ff-4349-b4de-e400c2e5a277 None admin] RBAC: Authorizing `identity:get_user(user_id=7a379f0e1f1f4c9c884081f682310ad9)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:27.953837 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09838fd7-43f3-4b4c-9631-dbb3395b38ee None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:27.955137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa70570-f4ff-4349-b4de-e400c2e5a277 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:27.958035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 79/166] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:27 2022] GET /identity/v3/users/7a379f0e1f1f4c9c884081f682310ad9 => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:27.959838 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cd97f-5d7d-466c-aa0a-f7335bf1b164 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:27.960380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cd97f-5d7d-466c-aa0a-f7335bf1b164 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:27.960624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 88/167] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:27 2022] GET /identity/v3/role_assignments?user.id=2a5cf324b6904a499ca6308498e53335&scope.project.id=5d18db22ee924d578a2f7079502940eb&role.id=08a7c861d84b43d7a4da431f31f3c2d7 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:27.961083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cd97f-5d7d-466c-aa0a-f7335bf1b164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:27.961757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09838fd7-43f3-4b4c-9631-dbb3395b38ee None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:27.985739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-513cd97f-5d7d-466c-aa0a-f7335bf1b164 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:27.987434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-513cd97f-5d7d-466c-aa0a-f7335bf1b164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:28.038447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:28.099823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:28.127618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09838fd7-43f3-4b4c-9631-dbb3395b38ee None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:28.153411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 80/168] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:28.154327 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cd97f-5d7d-466c-aa0a-f7335bf1b164 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:28.187020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907bf96-c54c-4bd4-8cdc-c39a15d8ab2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:28.187707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907bf96-c54c-4bd4-8cdc-c39a15d8ab2d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:28.187993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907bf96-c54c-4bd4-8cdc-c39a15d8ab2d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:28.188265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907bf96-c54c-4bd4-8cdc-c39a15d8ab2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:28.317289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907bf96-c54c-4bd4-8cdc-c39a15d8ab2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:28.318791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907bf96-c54c-4bd4-8cdc-c39a15d8ab2d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:28.438579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cd97f-5d7d-466c-aa0a-f7335bf1b164 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:28.458416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:28.498345 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb53e4-546f-4a9c-be27-afc936b4ab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:28.499340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb53e4-546f-4a9c-be27-afc936b4ab1e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:28.499822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb53e4-546f-4a9c-be27-afc936b4ab1e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:28.500315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb53e4-546f-4a9c-be27-afc936b4ab1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:28.508719 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 89/169] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:28.509643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907bf96-c54c-4bd4-8cdc-c39a15d8ab2d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:28.543474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:28.624571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb53e4-546f-4a9c-be27-afc936b4ab1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:28.627124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb53e4-546f-4a9c-be27-afc936b4ab1e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:28.753682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907bf96-c54c-4bd4-8cdc-c39a15d8ab2d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:28.763269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 81/170] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:28.769550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb53e4-546f-4a9c-be27-afc936b4ab1e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:28.835617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a06f94-7af2-4e7e-8d62-959bdbec1946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:28.836335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a06f94-7af2-4e7e-8d62-959bdbec1946 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:28.836524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a06f94-7af2-4e7e-8d62-959bdbec1946 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:28.836941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a06f94-7af2-4e7e-8d62-959bdbec1946 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:28.862658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a06f94-7af2-4e7e-8d62-959bdbec1946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f4a06f94-7af2-4e7e-8d62-959bdbec1946 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:28.874596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:28.875475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:28.876258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:28.876258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:28.876258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:28.876258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:28.876258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:28.876258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:28.876258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:28.876258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:28.876258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:28.876258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:28.876258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:28.876258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:28.876855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:28.876933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 90/171] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:28.876933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a06f94-7af2-4e7e-8d62-959bdbec1946 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:28.936903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb53e4-546f-4a9c-be27-afc936b4ab1e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:28.940386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:29.002986 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2fed3-5b6c-4dea-8422-c9c035aac1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:29.003620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2fed3-5b6c-4dea-8422-c9c035aac1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.004154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2fed3-5b6c-4dea-8422-c9c035aac1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.004430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2fed3-5b6c-4dea-8422-c9c035aac1e9 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.006459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2fed3-5b6c-4dea-8422-c9c035aac1e9 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:29.007922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2fed3-5b6c-4dea-8422-c9c035aac1e9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:29.010199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 82/172] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:28 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:29.010926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2fed3-5b6c-4dea-8422-c9c035aac1e9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:29.063469 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a06f94-7af2-4e7e-8d62-959bdbec1946 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:29.134436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb44b0-c0f7-4e02-9ecf-7b9b8a965b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:29.135203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb44b0-c0f7-4e02-9ecf-7b9b8a965b05 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.135401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb44b0-c0f7-4e02-9ecf-7b9b8a965b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.135673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb44b0-c0f7-4e02-9ecf-7b9b8a965b05 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.136909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb44b0-c0f7-4e02-9ecf-7b9b8a965b05 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:29.142651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb44b0-c0f7-4e02-9ecf-7b9b8a965b05 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:29.144393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 91/173] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:28 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:29.147254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f17f83-8b7d-4d52-8ed1-723420ca7a14 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.147534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f17f83-8b7d-4d52-8ed1-723420ca7a14 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.147806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f17f83-8b7d-4d52-8ed1-723420ca7a14 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.149044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 92/174] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:29.149763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f17f83-8b7d-4d52-8ed1-723420ca7a14 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:29.227289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2fed3-5b6c-4dea-8422-c9c035aac1e9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:29.281937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:29.290338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f17f83-8b7d-4d52-8ed1-723420ca7a14 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:29.293829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:29.302251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637229ed-7f4f-49ba-96af-0531f680e278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:29.302931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637229ed-7f4f-49ba-96af-0531f680e278 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.303213 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637229ed-7f4f-49ba-96af-0531f680e278 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.303485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637229ed-7f4f-49ba-96af-0531f680e278 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.304515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637229ed-7f4f-49ba-96af-0531f680e278 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:29.304874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637229ed-7f4f-49ba-96af-0531f680e278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:29.306398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637229ed-7f4f-49ba-96af-0531f680e278 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:29.326618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 83/175] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:29.330200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637229ed-7f4f-49ba-96af-0531f680e278 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:29.370615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b525c2f-ef13-4dbe-9167-078bdee6d63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:29.371240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b525c2f-ef13-4dbe-9167-078bdee6d63a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.371519 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b525c2f-ef13-4dbe-9167-078bdee6d63a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.371791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b525c2f-ef13-4dbe-9167-078bdee6d63a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.372527 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b525c2f-ef13-4dbe-9167-078bdee6d63a None admin] RBAC: Adding query filter params (role.id=1d1b5b96118040f9a4fd019bbcd93071, scope.project.id=5d18db22ee924d578a2f7079502940eb, user.id=2a5cf324b6904a499ca6308498e53335) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:29.372836 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b525c2f-ef13-4dbe-9167-078bdee6d63a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:29.374339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b525c2f-ef13-4dbe-9167-078bdee6d63a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:29.382668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 93/176] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:29 2022] GET /identity/v3/role_assignments?user.id=2a5cf324b6904a499ca6308498e53335&scope.project.id=5d18db22ee924d578a2f7079502940eb&role.id=1d1b5b96118040f9a4fd019bbcd93071 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:29.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85ce8-b440-44c0-a153-4b701a1de72b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.385110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85ce8-b440-44c0-a153-4b701a1de72b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.385508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85ce8-b440-44c0-a153-4b701a1de72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.420959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-45b85ce8-b440-44c0-a153-4b701a1de72b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:29.422653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-45b85ce8-b440-44c0-a153-4b701a1de72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:29.559756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637229ed-7f4f-49ba-96af-0531f680e278 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:29.578315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 94/177] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:29.579589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b85ce8-b440-44c0-a153-4b701a1de72b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:29.620049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba467906-a98d-40c3-8b2d-0305fe5f3fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:29.620659 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba467906-a98d-40c3-8b2d-0305fe5f3fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.620932 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba467906-a98d-40c3-8b2d-0305fe5f3fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.621195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba467906-a98d-40c3-8b2d-0305fe5f3fb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.621957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba467906-a98d-40c3-8b2d-0305fe5f3fb8 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=5d18db22ee924d578a2f7079502940eb, user.id=7a379f0e1f1f4c9c884081f682310ad9) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:29.622262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba467906-a98d-40c3-8b2d-0305fe5f3fb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:29.623697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba467906-a98d-40c3-8b2d-0305fe5f3fb8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:29.637760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 84/178] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:29 2022] GET /identity/v3/role_assignments?user.id=7a379f0e1f1f4c9c884081f682310ad9&scope.project.id=5d18db22ee924d578a2f7079502940eb&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:29.639267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984a75b-ddeb-4e89-8e45-c9c45f3e9a82 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.639556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984a75b-ddeb-4e89-8e45-c9c45f3e9a82 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.639828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984a75b-ddeb-4e89-8e45-c9c45f3e9a82 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.641163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 85/179] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:29.641938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984a75b-ddeb-4e89-8e45-c9c45f3e9a82 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:29.724276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:29.732449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b85ce8-b440-44c0-a153-4b701a1de72b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:29.734392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:29.774596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e653-1f17-4dcb-89c0-779e6f35d9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:29.775275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e653-1f17-4dcb-89c0-779e6f35d9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.775557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e653-1f17-4dcb-89c0-779e6f35d9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.775833 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e653-1f17-4dcb-89c0-779e6f35d9d8 None admin] PATH_INFO: `/v3/users/2a5cf324b6904a499ca6308498e53335` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.777005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e653-1f17-4dcb-89c0-779e6f35d9d8 None admin] RBAC: Authorizing `identity:get_user(user_id=2a5cf324b6904a499ca6308498e53335)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:29.778508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e653-1f17-4dcb-89c0-779e6f35d9d8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:29.780224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 95/180] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:29 2022] GET /identity/v3/users/2a5cf324b6904a499ca6308498e53335 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:29.781421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc1d7e-6876-4e3e-ad77-832f6ccca69f None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.781726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc1d7e-6876-4e3e-ad77-832f6ccca69f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.781996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc1d7e-6876-4e3e-ad77-832f6ccca69f None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.783460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 96/181] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:29.786888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350a42f-a81c-418d-a24b-2218335ef0ee None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.787364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350a42f-a81c-418d-a24b-2218335ef0ee None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.787646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350a42f-a81c-418d-a24b-2218335ef0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.790371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984a75b-ddeb-4e89-8e45-c9c45f3e9a82 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:29.808682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e350a42f-a81c-418d-a24b-2218335ef0ee None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:29.809938 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e350a42f-a81c-418d-a24b-2218335ef0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:29.831129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47d88e-b4cd-4a0b-a377-77db47f99e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:29.831762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47d88e-b4cd-4a0b-a377-77db47f99e3e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.832042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47d88e-b4cd-4a0b-a377-77db47f99e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.832317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47d88e-b4cd-4a0b-a377-77db47f99e3e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.845704 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47d88e-b4cd-4a0b-a377-77db47f99e3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-3b47d88e-b4cd-4a0b-a377-77db47f99e3e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:29.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:29.852310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:29.853153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 86/182] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:29.855094 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1dcca-596a-4884-b2cf-535ae49f032c None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.855094 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1dcca-596a-4884-b2cf-535ae49f032c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.855094 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1dcca-596a-4884-b2cf-535ae49f032c None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.855839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 87/183] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:29.856539 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1dcca-596a-4884-b2cf-535ae49f032c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:29.893705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 97/184] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:29.895267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4951e46-4d87-4535-80de-744d370231af None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:29.895753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4951e46-4d87-4535-80de-744d370231af None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:29.896095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4951e46-4d87-4535-80de-744d370231af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:29.914950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e4951e46-4d87-4535-80de-744d370231af None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:29.916682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e4951e46-4d87-4535-80de-744d370231af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:29.964763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1dcca-596a-4884-b2cf-535ae49f032c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:29.981829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:29.984226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 98/185] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:29.985086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4951e46-4d87-4535-80de-744d370231af None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:30.000486 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de00209-0a0a-4849-bffd-f0067116f3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:30.001152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de00209-0a0a-4849-bffd-f0067116f3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:30.001433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de00209-0a0a-4849-bffd-f0067116f3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:30.003603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de00209-0a0a-4849-bffd-f0067116f3f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:30.014659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:30.092765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4951e46-4d87-4535-80de-744d370231af None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:30.126338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fd696-ffc8-4fdb-b602-e6cd6b4d7650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:30.126945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fd696-ffc8-4fdb-b602-e6cd6b4d7650 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:30.127226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fd696-ffc8-4fdb-b602-e6cd6b4d7650 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:30.127498 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fd696-ffc8-4fdb-b602-e6cd6b4d7650 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:30.128163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fd696-ffc8-4fdb-b602-e6cd6b4d7650 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:30.128258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de00209-0a0a-4849-bffd-f0067116f3f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:30.128464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fd696-ffc8-4fdb-b602-e6cd6b4d7650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:30.129726 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de00209-0a0a-4849-bffd-f0067116f3f0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:30.129844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fd696-ffc8-4fdb-b602-e6cd6b4d7650 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:30.138401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 99/186] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:30.139557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9811efb-6d4b-4dff-aa8a-ed8afcb00eb6 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:30.139842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9811efb-6d4b-4dff-aa8a-ed8afcb00eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:30.140118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9811efb-6d4b-4dff-aa8a-ed8afcb00eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:30.158046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e9811efb-6d4b-4dff-aa8a-ed8afcb00eb6 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:30.159152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e9811efb-6d4b-4dff-aa8a-ed8afcb00eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:30.224173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 100/187] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:30.226424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9811efb-6d4b-4dff-aa8a-ed8afcb00eb6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:30.233270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 88/188] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:30.233270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de00209-0a0a-4849-bffd-f0067116f3f0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:30.248472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:30.249795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:30.330563 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9811efb-6d4b-4dff-aa8a-ed8afcb00eb6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:30.333879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de00209-0a0a-4849-bffd-f0067116f3f0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:30.364691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da471e37-a266-4686-a995-e687112e825e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:30.365334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da471e37-a266-4686-a995-e687112e825e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:30.365644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da471e37-a266-4686-a995-e687112e825e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:30.365916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da471e37-a266-4686-a995-e687112e825e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:30.377022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73052bc8-888d-476a-a431-50c6bdff5126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:30.377451 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da471e37-a266-4686-a995-e687112e825e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:30.377686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73052bc8-888d-476a-a431-50c6bdff5126 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:30.377969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73052bc8-888d-476a-a431-50c6bdff5126 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:30.378243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73052bc8-888d-476a-a431-50c6bdff5126 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:30.379390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73052bc8-888d-476a-a431-50c6bdff5126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:30.381106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73052bc8-888d-476a-a431-50c6bdff5126 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:30.382987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 89/189] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:30.383740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73052bc8-888d-476a-a431-50c6bdff5126 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-da471e37-a266-4686-a995-e687112e825e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:30.384123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:30.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 101/190] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:29:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:30.386617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da471e37-a266-4686-a995-e687112e825e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:30.466166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:30.472145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:30.486963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73052bc8-888d-476a-a431-50c6bdff5126 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:30.487716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da471e37-a266-4686-a995-e687112e825e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:30.519657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe9da1-8a6d-45e6-8bfb-2350283c7156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:30.520279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe9da1-8a6d-45e6-8bfb-2350283c7156 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:30.520552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe9da1-8a6d-45e6-8bfb-2350283c7156 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:30.520889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe9da1-8a6d-45e6-8bfb-2350283c7156 None admin] PATH_INFO: `/v3/users/450d6ceea9a743bdb53b50979b70d5c1` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:30.521845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79974e42-d0e8-4759-9aa4-87a1b1b64ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:30.522278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe9da1-8a6d-45e6-8bfb-2350283c7156 None admin] RBAC: Authorizing `identity:get_user(user_id=450d6ceea9a743bdb53b50979b70d5c1)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:30.522533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79974e42-d0e8-4759-9aa4-87a1b1b64ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:30.522750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79974e42-d0e8-4759-9aa4-87a1b1b64ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:30.523015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79974e42-d0e8-4759-9aa4-87a1b1b64ae4 None admin] PATH_INFO: `/v3/projects/11e833036a6d48439a0b123384a4ff79` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:30.524013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe9da1-8a6d-45e6-8bfb-2350283c7156 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:30.524121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79974e42-d0e8-4759-9aa4-87a1b1b64ae4 None admin] RBAC: Authorizing `identity:get_project(project_id=11e833036a6d48439a0b123384a4ff79)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:30.526135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79974e42-d0e8-4759-9aa4-87a1b1b64ae4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:30.526641 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 90/191] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:30 2022] GET /identity/v3/users/450d6ceea9a743bdb53b50979b70d5c1 => generated 289 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:30.527229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe9da1-8a6d-45e6-8bfb-2350283c7156 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:30.528004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 102/192] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:30 2022] GET /identity/v3/projects/11e833036a6d48439a0b123384a4ff79 => generated 312 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:30.528688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79974e42-d0e8-4759-9aa4-87a1b1b64ae4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:30.626290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe9da1-8a6d-45e6-8bfb-2350283c7156 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:30.628511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79974e42-d0e8-4759-9aa4-87a1b1b64ae4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:30.659695 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ace9b2-1df3-44de-b185-e01b91ad57a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:30.660313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ace9b2-1df3-44de-b185-e01b91ad57a1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:30.660588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ace9b2-1df3-44de-b185-e01b91ad57a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:30.660855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ace9b2-1df3-44de-b185-e01b91ad57a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:30.661127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98876603-47b6-4f14-9736-84d82a580ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:30.661751 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876603-47b6-4f14-9736-84d82a580ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:30.662033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876603-47b6-4f14-9736-84d82a580ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:30.662302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876603-47b6-4f14-9736-84d82a580ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:30.662957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98876603-47b6-4f14-9736-84d82a580ea4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:30.663259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98876603-47b6-4f14-9736-84d82a580ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:30.664684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98876603-47b6-4f14-9736-84d82a580ea4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:30.670848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ace9b2-1df3-44de-b185-e01b91ad57a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:30.672943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 103/193] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:30.673545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98876603-47b6-4f14-9736-84d82a580ea4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-17ace9b2-1df3-44de-b185-e01b91ad57a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:30.675649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:30.676683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:30.677475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 91/194] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:29:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:30.678182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ace9b2-1df3-44de-b185-e01b91ad57a1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:30.687115 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:30.693866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:30.777497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98876603-47b6-4f14-9736-84d82a580ea4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:30.779435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ace9b2-1df3-44de-b185-e01b91ad57a1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:30.809901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29d9e5-6aad-499e-b877-b7e941293103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:30.810503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29d9e5-6aad-499e-b877-b7e941293103 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:30.810781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29d9e5-6aad-499e-b877-b7e941293103 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:30.811052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29d9e5-6aad-499e-b877-b7e941293103 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:30.811689 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29d9e5-6aad-499e-b877-b7e941293103 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:30.812715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c947002-f8ed-4a46-9705-9aef1f599b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:30.813141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29d9e5-6aad-499e-b877-b7e941293103 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:30.813305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c947002-f8ed-4a46-9705-9aef1f599b8e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:30.813618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c947002-f8ed-4a46-9705-9aef1f599b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:30.813890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c947002-f8ed-4a46-9705-9aef1f599b8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:30.879310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c947002-f8ed-4a46-9705-9aef1f599b8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:30.880767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c947002-f8ed-4a46-9705-9aef1f599b8e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:30.895303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:30.895442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 104/195] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:29:30 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:29:30.896219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29d9e5-6aad-499e-b877-b7e941293103 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:30.964949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:31.007627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 92/196] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:31.007627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c947002-f8ed-4a46-9705-9aef1f599b8e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:31.007884 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29d9e5-6aad-499e-b877-b7e941293103 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:31.049640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d463db-a86b-42f3-998a-46bc4501265f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:31.049845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d463db-a86b-42f3-998a-46bc4501265f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:31.050059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d463db-a86b-42f3-998a-46bc4501265f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:31.050947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d463db-a86b-42f3-998a-46bc4501265f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:31.050947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d463db-a86b-42f3-998a-46bc4501265f None admin] RBAC: Adding query filter params (role.id=08a7c861d84b43d7a4da431f31f3c2d7, scope.project.id=11e833036a6d48439a0b123384a4ff79, user.id=2a5cf324b6904a499ca6308498e53335) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:31.051250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d463db-a86b-42f3-998a-46bc4501265f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:31.052706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d463db-a86b-42f3-998a-46bc4501265f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:31.060276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 105/197] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:30 2022] GET /identity/v3/role_assignments?user.id=2a5cf324b6904a499ca6308498e53335&scope.project.id=11e833036a6d48439a0b123384a4ff79&role.id=08a7c861d84b43d7a4da431f31f3c2d7 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:31.063924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d463db-a86b-42f3-998a-46bc4501265f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:31.150845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c947002-f8ed-4a46-9705-9aef1f599b8e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:31.171959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:31.197691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d463db-a86b-42f3-998a-46bc4501265f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:31.204185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd12edd-f0a5-4010-8025-aa5728f76ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:31.204856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd12edd-f0a5-4010-8025-aa5728f76ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:31.205136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd12edd-f0a5-4010-8025-aa5728f76ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:31.205415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd12edd-f0a5-4010-8025-aa5728f76ad7 None admin] PATH_INFO: `/v3/users/19468ccffc0146e188215f591f4ec198` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:31.224754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd12edd-f0a5-4010-8025-aa5728f76ad7 None admin] RBAC: Authorizing `identity:get_user(user_id=19468ccffc0146e188215f591f4ec198)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:31.226333 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd12edd-f0a5-4010-8025-aa5728f76ad7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:31.228073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 93/198] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:31 2022] GET /identity/v3/users/19468ccffc0146e188215f591f4ec198 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:31.229050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd12edd-f0a5-4010-8025-aa5728f76ad7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:31.233715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:31.249739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e10f5-840b-4d8a-a385-efbf90735a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:31.251115 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e10f5-840b-4d8a-a385-efbf90735a02 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:31.251115 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e10f5-840b-4d8a-a385-efbf90735a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:31.251115 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e10f5-840b-4d8a-a385-efbf90735a02 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:31.257004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e10f5-840b-4d8a-a385-efbf90735a02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:31.257004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e10f5-840b-4d8a-a385-efbf90735a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:31.257004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e10f5-840b-4d8a-a385-efbf90735a02 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:31.271145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 106/199] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:29:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:31.271145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e10f5-840b-4d8a-a385-efbf90735a02 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:31.369700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd12edd-f0a5-4010-8025-aa5728f76ad7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:31.410307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e10f5-840b-4d8a-a385-efbf90735a02 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:31.415305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80c3da-c51a-4a62-8fc0-99b7116e19f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:31.416664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80c3da-c51a-4a62-8fc0-99b7116e19f7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:31.417049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80c3da-c51a-4a62-8fc0-99b7116e19f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:31.417329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80c3da-c51a-4a62-8fc0-99b7116e19f7 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:31.418542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80c3da-c51a-4a62-8fc0-99b7116e19f7 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:31.420002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80c3da-c51a-4a62-8fc0-99b7116e19f7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:31.421832 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 94/200] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:31 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:31.422783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80c3da-c51a-4a62-8fc0-99b7116e19f7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:31.453871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26623e-cd97-4e12-89a5-fec796dfbc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:31.454485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26623e-cd97-4e12-89a5-fec796dfbc4e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:31.454761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26623e-cd97-4e12-89a5-fec796dfbc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:31.455028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26623e-cd97-4e12-89a5-fec796dfbc4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:31.489978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:31.521675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:31.534837 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26623e-cd97-4e12-89a5-fec796dfbc4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:31.536403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26623e-cd97-4e12-89a5-fec796dfbc4e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:31.544700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80c3da-c51a-4a62-8fc0-99b7116e19f7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:31.579625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e3543-a121-4c69-b01b-f08abd2d5fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:31.580516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e3543-a121-4c69-b01b-f08abd2d5fba None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:31.581959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e3543-a121-4c69-b01b-f08abd2d5fba None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:31.582304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e3543-a121-4c69-b01b-f08abd2d5fba None admin] PATH_INFO: `/v3/users/19468ccffc0146e188215f591f4ec198` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:31.583602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e3543-a121-4c69-b01b-f08abd2d5fba None admin] RBAC: Authorizing `identity:get_user(user_id=19468ccffc0146e188215f591f4ec198)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:31.585126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e3543-a121-4c69-b01b-f08abd2d5fba None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:31.591756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 95/201] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:31 2022] GET /identity/v3/users/19468ccffc0146e188215f591f4ec198 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:31.592710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e3543-a121-4c69-b01b-f08abd2d5fba None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:31.646856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 107/202] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:31.647761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26623e-cd97-4e12-89a5-fec796dfbc4e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:31.704166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e3543-a121-4c69-b01b-f08abd2d5fba None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:31.717673 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:31.748063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ac501-da00-494f-922c-155bae009f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:31.749056 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ac501-da00-494f-922c-155bae009f8e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:31.749580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ac501-da00-494f-922c-155bae009f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:31.750085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ac501-da00-494f-922c-155bae009f8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:31.750811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:31.753311 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ac501-da00-494f-922c-155bae009f8e None admin] RBAC: Adding query filter params (role.id=1b625e8349d04de08806337ee4bd9478, scope.project.id=5d18db22ee924d578a2f7079502940eb, user.id=450d6ceea9a743bdb53b50979b70d5c1) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:31.753921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ac501-da00-494f-922c-155bae009f8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:31.755577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ac501-da00-494f-922c-155bae009f8e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:31.763091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 96/203] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:31 2022] GET /identity/v3/role_assignments?user.id=450d6ceea9a743bdb53b50979b70d5c1&scope.project.id=5d18db22ee924d578a2f7079502940eb&role.id=1b625e8349d04de08806337ee4bd9478 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:31.764180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ac501-da00-494f-922c-155bae009f8e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:31.774792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26623e-cd97-4e12-89a5-fec796dfbc4e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:31.854368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb443c2c-c9ec-4035-b200-96e185314c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:31.855020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb443c2c-c9ec-4035-b200-96e185314c96 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:31.855406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb443c2c-c9ec-4035-b200-96e185314c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:31.855717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb443c2c-c9ec-4035-b200-96e185314c96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:31.911585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ac501-da00-494f-922c-155bae009f8e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:31.993385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb443c2c-c9ec-4035-b200-96e185314c96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:31.995797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb443c2c-c9ec-4035-b200-96e185314c96 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:32.000190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddce4c2-2ecc-40db-83e5-1861e3d48684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:32.001060 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddce4c2-2ecc-40db-83e5-1861e3d48684 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:32.002436 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddce4c2-2ecc-40db-83e5-1861e3d48684 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:32.003006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddce4c2-2ecc-40db-83e5-1861e3d48684 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:32.004998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddce4c2-2ecc-40db-83e5-1861e3d48684 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:32.006919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddce4c2-2ecc-40db-83e5-1861e3d48684 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:32.009285 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 97/204] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:31 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:32.015702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddce4c2-2ecc-40db-83e5-1861e3d48684 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:32.081562 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:32.116448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:32.138100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 108/205] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:32.145577 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb443c2c-c9ec-4035-b200-96e185314c96 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:32.168070 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddce4c2-2ecc-40db-83e5-1861e3d48684 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:32.216266 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf836f-0f18-45e4-90c6-c2eb9f6bccf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:32.216560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf836f-0f18-45e4-90c6-c2eb9f6bccf9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:32.216835 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf836f-0f18-45e4-90c6-c2eb9f6bccf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:32.217103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf836f-0f18-45e4-90c6-c2eb9f6bccf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:32.217870 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf836f-0f18-45e4-90c6-c2eb9f6bccf9 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85, user.id=19468ccffc0146e188215f591f4ec198) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:32.218174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf836f-0f18-45e4-90c6-c2eb9f6bccf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:32.219622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf836f-0f18-45e4-90c6-c2eb9f6bccf9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:32.226035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 98/206] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:32 2022] GET /identity/v3/role_assignments?user.id=19468ccffc0146e188215f591f4ec198&scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:32.270135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb443c2c-c9ec-4035-b200-96e185314c96 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:32.320521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3cdff-362e-4ae4-9fc7-e065fe29f461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:32.321215 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3cdff-362e-4ae4-9fc7-e065fe29f461 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:32.321553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3cdff-362e-4ae4-9fc7-e065fe29f461 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:32.321885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3cdff-362e-4ae4-9fc7-e065fe29f461 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:32.323903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3cdff-362e-4ae4-9fc7-e065fe29f461 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:32.325488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3cdff-362e-4ae4-9fc7-e065fe29f461 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:32.327813 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 109/207] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:32 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:32.333455 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf836f-0f18-45e4-90c6-c2eb9f6bccf9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:32.452395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf836f-0f18-45e4-90c6-c2eb9f6bccf9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:32.455711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:32.487894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9857743-3bb0-4b5b-b74d-542b888eda13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:32.488511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9857743-3bb0-4b5b-b74d-542b888eda13 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:32.488847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9857743-3bb0-4b5b-b74d-542b888eda13 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:32.489122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9857743-3bb0-4b5b-b74d-542b888eda13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:32.489948 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9857743-3bb0-4b5b-b74d-542b888eda13 None admin] RBAC: Adding query filter params (role.id=1d1b5b96118040f9a4fd019bbcd93071, scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85, user.id=19468ccffc0146e188215f591f4ec198) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:32.490259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9857743-3bb0-4b5b-b74d-542b888eda13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:32.491710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9857743-3bb0-4b5b-b74d-542b888eda13 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:32.497257 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 99/208] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:32 2022] GET /identity/v3/role_assignments?user.id=19468ccffc0146e188215f591f4ec198&scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85&role.id=1d1b5b96118040f9a4fd019bbcd93071 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:33.252816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d49e25-5c7c-4f4e-9d92-497f13f419bd None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:33.253176 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d49e25-5c7c-4f4e-9d92-497f13f419bd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:33.253396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d49e25-5c7c-4f4e-9d92-497f13f419bd None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:33.254978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 110/209] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:33.265603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e1837-22ab-44d4-9cba-3e5e8307df1f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:33.265603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e1837-22ab-44d4-9cba-3e5e8307df1f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:33.265603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e1837-22ab-44d4-9cba-3e5e8307df1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:33.294061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b01e1837-22ab-44d4-9cba-3e5e8307df1f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:33.295566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b01e1837-22ab-44d4-9cba-3e5e8307df1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:33.347941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca552c8-c3b3-47fb-bfa7-ec50174e1c78 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:33.347941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca552c8-c3b3-47fb-bfa7-ec50174e1c78 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:33.347941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca552c8-c3b3-47fb-bfa7-ec50174e1c78 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:33.348894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 111/210] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:33.354853 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dec39d-1ede-413e-bad2-6a166140214a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:33.355403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dec39d-1ede-413e-bad2-6a166140214a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:33.355908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dec39d-1ede-413e-bad2-6a166140214a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:33.367971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 100/211] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:33.375250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e1dec39d-1ede-413e-bad2-6a166140214a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:33.376759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e1dec39d-1ede-413e-bad2-6a166140214a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:33.393812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e1837-22ab-44d4-9cba-3e5e8307df1f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:33.438717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:33.457374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 112/212] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:33.481730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dec39d-1ede-413e-bad2-6a166140214a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:33.526787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:33.531197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e1837-22ab-44d4-9cba-3e5e8307df1f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:33.583069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31006251-6a7f-4b51-ab02-5e79a7830b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:33.583767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31006251-6a7f-4b51-ab02-5e79a7830b86 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:33.584301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31006251-6a7f-4b51-ab02-5e79a7830b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:33.584764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31006251-6a7f-4b51-ab02-5e79a7830b86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:33.596103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dec39d-1ede-413e-bad2-6a166140214a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:33.601577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31006251-6a7f-4b51-ab02-5e79a7830b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-31006251-6a7f-4b51-ab02-5e79a7830b86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:33.606891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:33.607759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:33.608543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:33.608543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:33.608543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:33.608543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:33.608543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:33.608543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:33.608543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:33.608543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:33.608543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:33.608543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:33.608543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:33.608543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:33.611050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 101/213] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:33.617031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31006251-6a7f-4b51-ab02-5e79a7830b86 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:33.631819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd7bb2-2f3a-4af9-be70-f8e0f6d64854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:33.632789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd7bb2-2f3a-4af9-be70-f8e0f6d64854 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:33.633249 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd7bb2-2f3a-4af9-be70-f8e0f6d64854 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:33.633772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd7bb2-2f3a-4af9-be70-f8e0f6d64854 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:33.635751 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd7bb2-2f3a-4af9-be70-f8e0f6d64854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:33.637483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd7bb2-2f3a-4af9-be70-f8e0f6d64854 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:33.639776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 113/214] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:33.645377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd7bb2-2f3a-4af9-be70-f8e0f6d64854 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:33.723452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:33.747429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31006251-6a7f-4b51-ab02-5e79a7830b86 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:33.780686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd7bb2-2f3a-4af9-be70-f8e0f6d64854 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:33.792074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf4780-04cc-4402-8f91-4b36c36b10a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:33.792944 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf4780-04cc-4402-8f91-4b36c36b10a5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:33.793436 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf4780-04cc-4402-8f91-4b36c36b10a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:33.793917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf4780-04cc-4402-8f91-4b36c36b10a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:33.794767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf4780-04cc-4402-8f91-4b36c36b10a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:33.795264 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf4780-04cc-4402-8f91-4b36c36b10a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:33.796809 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf4780-04cc-4402-8f91-4b36c36b10a5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:33.811847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 102/215] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:33.812983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf4780-04cc-4402-8f91-4b36c36b10a5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:33.825840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359309b3-dc82-43a6-94ff-a848069f7e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:33.826931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359309b3-dc82-43a6-94ff-a848069f7e05 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:33.827223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359309b3-dc82-43a6-94ff-a848069f7e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:33.827504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359309b3-dc82-43a6-94ff-a848069f7e05 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:33.828166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359309b3-dc82-43a6-94ff-a848069f7e05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:33.830197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359309b3-dc82-43a6-94ff-a848069f7e05 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:33.838838 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:33.917699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 114/216] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:29:33 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:29:33.930057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf4780-04cc-4402-8f91-4b36c36b10a5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:33.971054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96226de-8048-450a-85f3-5dfe27471277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:33.972059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96226de-8048-450a-85f3-5dfe27471277 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:33.972539 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96226de-8048-450a-85f3-5dfe27471277 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:33.973011 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96226de-8048-450a-85f3-5dfe27471277 None admin] PATH_INFO: `/v3/users/34dbeffb7d6c4574bcfed5188e835175` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:33.988929 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96226de-8048-450a-85f3-5dfe27471277 None admin] RBAC: Authorizing `identity:get_user(user_id=34dbeffb7d6c4574bcfed5188e835175)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:33.990883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96226de-8048-450a-85f3-5dfe27471277 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:33.993226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 103/217] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:33 2022] GET /identity/v3/users/34dbeffb7d6c4574bcfed5188e835175 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:33.999300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359309b3-dc82-43a6-94ff-a848069f7e05 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:34.120504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359309b3-dc82-43a6-94ff-a848069f7e05 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:34.132037 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:34.157268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc7f22-3c08-4645-95ca-d7d98d191fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:34.157936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc7f22-3c08-4645-95ca-d7d98d191fae None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:34.158221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc7f22-3c08-4645-95ca-d7d98d191fae None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:34.158495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc7f22-3c08-4645-95ca-d7d98d191fae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:34.233315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc7f22-3c08-4645-95ca-d7d98d191fae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:34.235091 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc7f22-3c08-4645-95ca-d7d98d191fae None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:34.329542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dc126-8b2b-43ef-928b-a585dee58283 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:34.330046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dc126-8b2b-43ef-928b-a585dee58283 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:34.330527 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dc126-8b2b-43ef-928b-a585dee58283 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:34.332431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 104/218] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:29:34.338003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5e551-8892-4c54-bc6d-2fa70c4ab480 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:34.338003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5e551-8892-4c54-bc6d-2fa70c4ab480 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:34.338735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5e551-8892-4c54-bc6d-2fa70c4ab480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:34.358109 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-43c5e551-8892-4c54-bc6d-2fa70c4ab480 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:34.359347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-43c5e551-8892-4c54-bc6d-2fa70c4ab480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:34.366895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 115/219] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:34.373607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc7f22-3c08-4645-95ca-d7d98d191fae None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:34.386764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:34.450725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 105/220] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:34.490612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5e551-8892-4c54-bc6d-2fa70c4ab480 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:34.507382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc7f22-3c08-4645-95ca-d7d98d191fae None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:34.513871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:34.547208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa06c6-a18c-4c26-86e0-f036d12289b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:34.547902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa06c6-a18c-4c26-86e0-f036d12289b0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:34.548188 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa06c6-a18c-4c26-86e0-f036d12289b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:34.548469 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa06c6-a18c-4c26-86e0-f036d12289b0 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:34.549644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa06c6-a18c-4c26-86e0-f036d12289b0 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:34.551110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa06c6-a18c-4c26-86e0-f036d12289b0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:34.553686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 116/221] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:34 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:34.560595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa06c6-a18c-4c26-86e0-f036d12289b0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:34.678629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5e551-8892-4c54-bc6d-2fa70c4ab480 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:34.694362 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:34.718341 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa06c6-a18c-4c26-86e0-f036d12289b0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:34.743450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc851cc-3d2b-4442-a52d-7cd20ec6100f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:34.745485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc851cc-3d2b-4442-a52d-7cd20ec6100f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:34.746313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc851cc-3d2b-4442-a52d-7cd20ec6100f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:34.746811 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc851cc-3d2b-4442-a52d-7cd20ec6100f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:34.760210 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4c96-49f4-4ad4-853a-23ec877d8713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:34.761093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4c96-49f4-4ad4-853a-23ec877d8713 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:34.761381 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4c96-49f4-4ad4-853a-23ec877d8713 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:34.761672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4c96-49f4-4ad4-853a-23ec877d8713 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:34.762131 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc851cc-3d2b-4442-a52d-7cd20ec6100f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:34.762411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e4c96-49f4-4ad4-853a-23ec877d8713 None admin] RBAC: Adding query filter params (role.id=08a7c861d84b43d7a4da431f31f3c2d7, scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85, user.id=34dbeffb7d6c4574bcfed5188e835175) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:34.762718 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e4c96-49f4-4ad4-853a-23ec877d8713 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:34.764174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e4c96-49f4-4ad4-853a-23ec877d8713 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-0fc851cc-3d2b-4442-a52d-7cd20ec6100f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:34.768792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:34.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:34.771051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:34.771051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:34.771051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:34.771051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:34.771051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:34.771051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:34.771051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:34.771051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:34.771051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:34.771051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:34.771051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:34.771051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:34.771660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 117/222] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:34 2022] GET /identity/v3/role_assignments?user.id=34dbeffb7d6c4574bcfed5188e835175&scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85&role.id=08a7c861d84b43d7a4da431f31f3c2d7 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:34.771660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f61f4-9710-4e6f-a78e-608eb912f3cb None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:34.771660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f61f4-9710-4e6f-a78e-608eb912f3cb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:34.772114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f61f4-9710-4e6f-a78e-608eb912f3cb None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:34.773386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 118/223] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:34.773797 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 106/224] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:29:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:34.779316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120be36a-db19-4a9d-b2f1-2b561670df20 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:34.779761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc851cc-3d2b-4442-a52d-7cd20ec6100f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:34.780005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120be36a-db19-4a9d-b2f1-2b561670df20 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:34.780555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120be36a-db19-4a9d-b2f1-2b561670df20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:34.800093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-120be36a-db19-4a9d-b2f1-2b561670df20 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:34.801629 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-120be36a-db19-4a9d-b2f1-2b561670df20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:34.857875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:34.879108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 119/225] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:34.896402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc851cc-3d2b-4442-a52d-7cd20ec6100f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:34.913652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120be36a-db19-4a9d-b2f1-2b561670df20 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:34.940879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ac251-d3df-46e1-ba65-1246920b2ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:34.941423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ac251-d3df-46e1-ba65-1246920b2ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:34.942566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ac251-d3df-46e1-ba65-1246920b2ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:34.942839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ac251-d3df-46e1-ba65-1246920b2ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:34.943531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ac251-d3df-46e1-ba65-1246920b2ec6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:34.943832 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ac251-d3df-46e1-ba65-1246920b2ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:34.945231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ac251-d3df-46e1-ba65-1246920b2ec6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:34.961032 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 107/226] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:34.967291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ac251-d3df-46e1-ba65-1246920b2ec6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:35.019170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:35.057202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120be36a-db19-4a9d-b2f1-2b561670df20 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:35.109255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b859b7-7969-4710-9195-06f8992e2270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:35.110092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b859b7-7969-4710-9195-06f8992e2270 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:35.110373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b859b7-7969-4710-9195-06f8992e2270 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:35.110643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b859b7-7969-4710-9195-06f8992e2270 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:35.123309 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b859b7-7969-4710-9195-06f8992e2270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-d4b859b7-7969-4710-9195-06f8992e2270 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:35.128986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:35.129922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:35.131006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:35.131006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:35.131006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:35.131006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:35.131006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:35.131006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:35.131006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:35.131006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:35.131006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:35.131006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:35.131006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:35.131006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:35.131745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ac251-d3df-46e1-ba65-1246920b2ec6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:35.131828 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 120/227] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:29:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:35.138528 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b859b7-7969-4710-9195-06f8992e2270 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:35.174806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:35.176042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaf7a8-556d-4c36-ac8f-62d9269aff43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:35.176740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaf7a8-556d-4c36-ac8f-62d9269aff43 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:35.177095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaf7a8-556d-4c36-ac8f-62d9269aff43 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:35.177589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaf7a8-556d-4c36-ac8f-62d9269aff43 None admin] PATH_INFO: `/v3/users/19468ccffc0146e188215f591f4ec198` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:35.178812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaf7a8-556d-4c36-ac8f-62d9269aff43 None admin] RBAC: Authorizing `identity:get_user(user_id=19468ccffc0146e188215f591f4ec198)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:35.180682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaf7a8-556d-4c36-ac8f-62d9269aff43 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:35.182814 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 108/228] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:34 2022] GET /identity/v3/users/19468ccffc0146e188215f591f4ec198 => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:35.188459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaf7a8-556d-4c36-ac8f-62d9269aff43 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:35.327817 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaf7a8-556d-4c36-ac8f-62d9269aff43 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:35.363048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ec545-95fd-4df5-ad73-d407a0f8ea80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:35.363685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ec545-95fd-4df5-ad73-d407a0f8ea80 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:35.363966 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ec545-95fd-4df5-ad73-d407a0f8ea80 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:35.364240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ec545-95fd-4df5-ad73-d407a0f8ea80 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:35.364670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b859b7-7969-4710-9195-06f8992e2270 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:35.365378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ec545-95fd-4df5-ad73-d407a0f8ea80 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:35.369267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ec545-95fd-4df5-ad73-d407a0f8ea80 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:35.369267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 109/229] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:35 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:35.381931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ec545-95fd-4df5-ad73-d407a0f8ea80 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:35.430822 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:35.444772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce71737-57b6-419d-a955-1657e1cb15ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:35.448914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce71737-57b6-419d-a955-1657e1cb15ae None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:35.448914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce71737-57b6-419d-a955-1657e1cb15ae None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:35.448914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce71737-57b6-419d-a955-1657e1cb15ae None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:35.448914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce71737-57b6-419d-a955-1657e1cb15ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:35.448914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce71737-57b6-419d-a955-1657e1cb15ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:35.449831 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce71737-57b6-419d-a955-1657e1cb15ae None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:35.457546 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:35.470712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 121/230] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:29:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:35.477507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce71737-57b6-419d-a955-1657e1cb15ae None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:35.518939 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ec545-95fd-4df5-ad73-d407a0f8ea80 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:35.585040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0871e-97dd-4e04-817b-54c1ba328b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:35.588493 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0871e-97dd-4e04-817b-54c1ba328b3b None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:35.588778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0871e-97dd-4e04-817b-54c1ba328b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:35.589055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0871e-97dd-4e04-817b-54c1ba328b3b None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85/users/19468ccffc0146e188215f591f4ec198/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:35.591782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0871e-97dd-4e04-817b-54c1ba328b3b None admin] RBAC: Authorizing `identity:create_grant(project_id=38ed82c9c6da41e596c28a3fb28c8c85, user_id=19468ccffc0146e188215f591f4ec198, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:35.593250 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0871e-97dd-4e04-817b-54c1ba328b3b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:35.647187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce71737-57b6-419d-a955-1657e1cb15ae None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:35.682957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854da99-c965-4ca1-9432-c1717c4ef285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:35.683634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854da99-c965-4ca1-9432-c1717c4ef285 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:35.683960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854da99-c965-4ca1-9432-c1717c4ef285 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:35.684240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854da99-c965-4ca1-9432-c1717c4ef285 None admin] PATH_INFO: `/v3/users/19468ccffc0146e188215f591f4ec198` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:35.685372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854da99-c965-4ca1-9432-c1717c4ef285 None admin] RBAC: Authorizing `identity:get_user(user_id=19468ccffc0146e188215f591f4ec198)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:35.686857 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854da99-c965-4ca1-9432-c1717c4ef285 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:35.688928 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 122/231] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:35 2022] GET /identity/v3/users/19468ccffc0146e188215f591f4ec198 => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:35.694473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854da99-c965-4ca1-9432-c1717c4ef285 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:35.723615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 110/232] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:29:35 2022] PUT /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85/users/19468ccffc0146e188215f591f4ec198/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:29:35.779766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:35.821491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854da99-c965-4ca1-9432-c1717c4ef285 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:35.865681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcf68f-560d-4be0-a8a7-3233c4bcc2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:35.866253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcf68f-560d-4be0-a8a7-3233c4bcc2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:35.866536 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcf68f-560d-4be0-a8a7-3233c4bcc2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:35.866875 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcf68f-560d-4be0-a8a7-3233c4bcc2e1 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:35.868093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcf68f-560d-4be0-a8a7-3233c4bcc2e1 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:35.870147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcf68f-560d-4be0-a8a7-3233c4bcc2e1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:35.872077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 123/233] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:35 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:35.878164 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0871e-97dd-4e04-817b-54c1ba328b3b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:35.996771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0871e-97dd-4e04-817b-54c1ba328b3b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:36.024787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:36.033558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97318dd2-0203-4224-b504-ef806f6694a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:36.034237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97318dd2-0203-4224-b504-ef806f6694a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:36.034522 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97318dd2-0203-4224-b504-ef806f6694a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:36.034801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97318dd2-0203-4224-b504-ef806f6694a2 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85/users/19468ccffc0146e188215f591f4ec198/roles/1d1b5b96118040f9a4fd019bbcd93071` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:36.037021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97318dd2-0203-4224-b504-ef806f6694a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=38ed82c9c6da41e596c28a3fb28c8c85, user_id=19468ccffc0146e188215f591f4ec198, role_id=1d1b5b96118040f9a4fd019bbcd93071)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:36.038501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97318dd2-0203-4224-b504-ef806f6694a2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:36.131424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 111/234] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:29:35 2022] PUT /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85/users/19468ccffc0146e188215f591f4ec198/roles/1d1b5b96118040f9a4fd019bbcd93071 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:29:36.554015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e235c3-7f97-48b5-9d8c-2649ab5e437f None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:36.554773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e235c3-7f97-48b5-9d8c-2649ab5e437f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:36.554773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e235c3-7f97-48b5-9d8c-2649ab5e437f None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:36.556753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 124/235] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:36.562333 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9f0f4-7dae-4904-813f-b4280569fe17 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:36.562626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9f0f4-7dae-4904-813f-b4280569fe17 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:36.562903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9f0f4-7dae-4904-813f-b4280569fe17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:36.583466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-83e9f0f4-7dae-4904-813f-b4280569fe17 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:36.584963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-83e9f0f4-7dae-4904-813f-b4280569fe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:36.622978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8869cb-741e-4e4d-87f9-adeeaabfe073 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:36.623283 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8869cb-741e-4e4d-87f9-adeeaabfe073 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:36.623524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8869cb-741e-4e4d-87f9-adeeaabfe073 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:36.624869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 125/236] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:29:36.635505 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09066e-40c3-4d8e-9192-84e531f3379a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:36.635775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09066e-40c3-4d8e-9192-84e531f3379a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:36.636047 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09066e-40c3-4d8e-9192-84e531f3379a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:36.668634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8b09066e-40c3-4d8e-9192-84e531f3379a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:36.671387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8b09066e-40c3-4d8e-9192-84e531f3379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:36.682688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 112/237] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:36.705422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9f0f4-7dae-4904-813f-b4280569fe17 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:36.815846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 126/238] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:36.850865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09066e-40c3-4d8e-9192-84e531f3379a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:36.881734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9f0f4-7dae-4904-813f-b4280569fe17 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:36.921170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:36.952402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:36.962226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faf25e-2a8d-4f3c-9d90-436bc6447ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:36.962911 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faf25e-2a8d-4f3c-9d90-436bc6447ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:36.963195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faf25e-2a8d-4f3c-9d90-436bc6447ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:36.963744 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faf25e-2a8d-4f3c-9d90-436bc6447ac1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:36.991954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faf25e-2a8d-4f3c-9d90-436bc6447ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-a2faf25e-2a8d-4f3c-9d90-436bc6447ac1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:36.997306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:36.998212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:36.998987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 113/239] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:37.006352 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faf25e-2a8d-4f3c-9d90-436bc6447ac1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:37.009516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09066e-40c3-4d8e-9192-84e531f3379a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:37.050656 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275e78f-80a4-40c9-8526-b8d9593c2f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:37.051637 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275e78f-80a4-40c9-8526-b8d9593c2f07 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:37.052184 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275e78f-80a4-40c9-8526-b8d9593c2f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:37.052652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275e78f-80a4-40c9-8526-b8d9593c2f07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:37.054116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275e78f-80a4-40c9-8526-b8d9593c2f07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:37.057581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275e78f-80a4-40c9-8526-b8d9593c2f07 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:37.060186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 127/240] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:37.065488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275e78f-80a4-40c9-8526-b8d9593c2f07 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:37.165448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faf25e-2a8d-4f3c-9d90-436bc6447ac1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:37.203570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212a853-81e4-4539-929d-a9bd48913b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:37.204515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212a853-81e4-4539-929d-a9bd48913b69 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:37.204991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212a853-81e4-4539-929d-a9bd48913b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:37.205476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212a853-81e4-4539-929d-a9bd48913b69 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:37.206361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212a853-81e4-4539-929d-a9bd48913b69 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:37.206859 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212a853-81e4-4539-929d-a9bd48913b69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:37.208435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212a853-81e4-4539-929d-a9bd48913b69 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:37.217874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 114/241] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:37.223711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeba3d-5209-4c76-a1d6-3d6d8082593d None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:37.224287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeba3d-5209-4c76-a1d6-3d6d8082593d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:37.224774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeba3d-5209-4c76-a1d6-3d6d8082593d None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:37.226581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 115/242] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:37.226686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275e78f-80a4-40c9-8526-b8d9593c2f07 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:37.227588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdeba3d-5209-4c76-a1d6-3d6d8082593d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:37.271837 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:37.281761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68c4fb-5dc8-407f-9707-e7fe1683c67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:37.282438 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68c4fb-5dc8-407f-9707-e7fe1683c67b None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:37.282739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68c4fb-5dc8-407f-9707-e7fe1683c67b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:37.283005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68c4fb-5dc8-407f-9707-e7fe1683c67b None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:37.283637 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68c4fb-5dc8-407f-9707-e7fe1683c67b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:37.285453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68c4fb-5dc8-407f-9707-e7fe1683c67b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:37.285951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:37.359269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdeba3d-5209-4c76-a1d6-3d6d8082593d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:37.397476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dd452-dd45-4734-92a7-bd300f51241f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:37.398577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dd452-dd45-4734-92a7-bd300f51241f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:37.399117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dd452-dd45-4734-92a7-bd300f51241f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:37.399591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dd452-dd45-4734-92a7-bd300f51241f None admin] PATH_INFO: `/v3/users/dc810d059ad14efe91729d2b7913a166` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:37.401822 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 128/243] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:29:37 2022] POST /identity/v3/users => generated 293 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:29:37.404171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c51c3-cf1c-4534-8919-52ccb542d64d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:37.404473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c51c3-cf1c-4534-8919-52ccb542d64d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:37.404743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c51c3-cf1c-4534-8919-52ccb542d64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:37.420405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dd452-dd45-4734-92a7-bd300f51241f None admin] RBAC: Authorizing `identity:get_user(user_id=dc810d059ad14efe91729d2b7913a166)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:37.422584 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dd452-dd45-4734-92a7-bd300f51241f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:37.424760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 116/244] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:37 2022] GET /identity/v3/users/dc810d059ad14efe91729d2b7913a166 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:37.431448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dd452-dd45-4734-92a7-bd300f51241f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:37.455100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c18c51c3-cf1c-4534-8919-52ccb542d64d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:37.456375 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c18c51c3-cf1c-4534-8919-52ccb542d64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:37.548327 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:37.572585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dd452-dd45-4734-92a7-bd300f51241f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:37.583178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 129/245] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:37.613149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c51c3-cf1c-4534-8919-52ccb542d64d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:37.636609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383a57a-204a-4604-b55f-1b3cae75afca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:37.637954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383a57a-204a-4604-b55f-1b3cae75afca None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:37.639205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383a57a-204a-4604-b55f-1b3cae75afca None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:37.639876 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383a57a-204a-4604-b55f-1b3cae75afca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:37.723370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383a57a-204a-4604-b55f-1b3cae75afca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:37.725302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383a57a-204a-4604-b55f-1b3cae75afca None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:37.748440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c51c3-cf1c-4534-8919-52ccb542d64d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:37.763423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:37.786409 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9baa4c-f424-48bf-a6c6-79cd3b490542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:37.787060 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9baa4c-f424-48bf-a6c6-79cd3b490542 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:37.787350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9baa4c-f424-48bf-a6c6-79cd3b490542 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:37.787623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9baa4c-f424-48bf-a6c6-79cd3b490542 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:37.798896 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9baa4c-f424-48bf-a6c6-79cd3b490542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-0d9baa4c-f424-48bf-a6c6-79cd3b490542 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:37.803806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:37.805186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:37.806727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:37.806727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:37.806727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:37.806727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:37.806727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:37.806727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:37.806727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:37.806727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:37.806727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:37.806727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:37.806727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:37.806727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:37.809199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 130/246] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:37.819174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9baa4c-f424-48bf-a6c6-79cd3b490542 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:37.845878 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:37.854671 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 117/247] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:37.861410 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383a57a-204a-4604-b55f-1b3cae75afca None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:37.974704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9baa4c-f424-48bf-a6c6-79cd3b490542 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:37.994879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383a57a-204a-4604-b55f-1b3cae75afca None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:38.022573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476dd17-7f62-4c52-ad02-8301e824920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:38.023477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476dd17-7f62-4c52-ad02-8301e824920e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:38.027791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476dd17-7f62-4c52-ad02-8301e824920e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:38.027791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476dd17-7f62-4c52-ad02-8301e824920e None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:38.027791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476dd17-7f62-4c52-ad02-8301e824920e None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:38.027791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476dd17-7f62-4c52-ad02-8301e824920e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:38.028645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476dd17-7f62-4c52-ad02-8301e824920e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:38.042044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 131/248] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:38.044466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5625bab-76f4-4807-ab5e-f5c1c9b73f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:38.045353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5625bab-76f4-4807-ab5e-f5c1c9b73f77 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:38.045869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5625bab-76f4-4807-ab5e-f5c1c9b73f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:38.046344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5625bab-76f4-4807-ab5e-f5c1c9b73f77 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:38.047710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5625bab-76f4-4807-ab5e-f5c1c9b73f77 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:38.048808 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476dd17-7f62-4c52-ad02-8301e824920e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:38.051323 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5625bab-76f4-4807-ab5e-f5c1c9b73f77 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:38.056144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 118/249] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:37 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:38.069259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5625bab-76f4-4807-ab5e-f5c1c9b73f77 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:38.095074 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:38.156080 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:38.183729 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476dd17-7f62-4c52-ad02-8301e824920e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:38.201110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5625bab-76f4-4807-ab5e-f5c1c9b73f77 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:38.225503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84d297-d675-4896-937c-356e67b58538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:38.225830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84d297-d675-4896-937c-356e67b58538 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:38.226057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84d297-d675-4896-937c-356e67b58538 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:38.226331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84d297-d675-4896-937c-356e67b58538 None admin] PATH_INFO: `/v3/users/34dbeffb7d6c4574bcfed5188e835175` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:38.227508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84d297-d675-4896-937c-356e67b58538 None admin] RBAC: Authorizing `identity:get_user(user_id=34dbeffb7d6c4574bcfed5188e835175)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:38.230642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84d297-d675-4896-937c-356e67b58538 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:38.232118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 132/250] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:38 2022] GET /identity/v3/users/34dbeffb7d6c4574bcfed5188e835175 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:38.238839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84d297-d675-4896-937c-356e67b58538 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:38.251059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20837b6a-32df-4c5f-bf79-dc33aa275e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:38.252217 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20837b6a-32df-4c5f-bf79-dc33aa275e83 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:38.252787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20837b6a-32df-4c5f-bf79-dc33aa275e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:38.253367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20837b6a-32df-4c5f-bf79-dc33aa275e83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:38.254665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20837b6a-32df-4c5f-bf79-dc33aa275e83 None admin] RBAC: Adding query filter params (role.id=1b625e8349d04de08806337ee4bd9478, scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85, user.id=dc810d059ad14efe91729d2b7913a166) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:38.255388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20837b6a-32df-4c5f-bf79-dc33aa275e83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:38.258457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20837b6a-32df-4c5f-bf79-dc33aa275e83 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:38.274415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 119/251] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:38 2022] GET /identity/v3/role_assignments?user.id=dc810d059ad14efe91729d2b7913a166&scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85&role.id=1b625e8349d04de08806337ee4bd9478 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:38.368378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc52a8-1991-41e8-8005-d909d1e4295e None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:38.369608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc52a8-1991-41e8-8005-d909d1e4295e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:38.370022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc52a8-1991-41e8-8005-d909d1e4295e None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:38.371512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 120/252] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:29:38.374256 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84d297-d675-4896-937c-356e67b58538 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:38.375507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:38.378163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569f2a7-eeab-4192-a897-3dd73ae1403f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:38.378609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569f2a7-eeab-4192-a897-3dd73ae1403f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:38.379085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569f2a7-eeab-4192-a897-3dd73ae1403f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:38.414477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a0322-e1f6-4233-a583-a17aa50aa500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:38.415961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a0322-e1f6-4233-a583-a17aa50aa500 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:38.416353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a0322-e1f6-4233-a583-a17aa50aa500 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:38.416724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a0322-e1f6-4233-a583-a17aa50aa500 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:38.418075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a0322-e1f6-4233-a583-a17aa50aa500 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:38.421548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a0322-e1f6-4233-a583-a17aa50aa500 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:38.421548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 133/253] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:38 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:38.422810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0569f2a7-eeab-4192-a897-3dd73ae1403f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:38.424219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0569f2a7-eeab-4192-a897-3dd73ae1403f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:38.427661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a0322-e1f6-4233-a583-a17aa50aa500 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:38.556931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a0322-e1f6-4233-a583-a17aa50aa500 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:38.562568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 121/254] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:38.593939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8111953-bcd6-4272-9ce4-34116f21ed5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:38.594754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8111953-bcd6-4272-9ce4-34116f21ed5f None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:38.595136 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8111953-bcd6-4272-9ce4-34116f21ed5f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:38.595512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8111953-bcd6-4272-9ce4-34116f21ed5f None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85/users/34dbeffb7d6c4574bcfed5188e835175/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:38.597477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8111953-bcd6-4272-9ce4-34116f21ed5f None admin] RBAC: Authorizing `identity:create_grant(project_id=38ed82c9c6da41e596c28a3fb28c8c85, user_id=34dbeffb7d6c4574bcfed5188e835175, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:38.599002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8111953-bcd6-4272-9ce4-34116f21ed5f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:38.606379 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569f2a7-eeab-4192-a897-3dd73ae1403f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:38.668484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 134/255] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:29:38 2022] PUT /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85/users/34dbeffb7d6c4574bcfed5188e835175/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:29:38.675594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0737c10-9e19-4172-8d06-e8f3ed7d23d3 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:38.675906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0737c10-9e19-4172-8d06-e8f3ed7d23d3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:38.676149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0737c10-9e19-4172-8d06-e8f3ed7d23d3 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:38.677594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 135/256] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:38.684132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12218879-ea0d-4eff-9046-c93948333b0b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:38.684425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12218879-ea0d-4eff-9046-c93948333b0b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:38.684694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12218879-ea0d-4eff-9046-c93948333b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:38.690618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:38.715465 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-12218879-ea0d-4eff-9046-c93948333b0b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:38.716918 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-12218879-ea0d-4eff-9046-c93948333b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:38.757933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:38.804694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569f2a7-eeab-4192-a897-3dd73ae1403f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:38.835775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 136/257] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:38.861058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12218879-ea0d-4eff-9046-c93948333b0b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:38.891019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e9bee-04d5-41ca-8b64-bbc0753bf815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:38.891878 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e9bee-04d5-41ca-8b64-bbc0753bf815 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:38.892286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e9bee-04d5-41ca-8b64-bbc0753bf815 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:38.892751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e9bee-04d5-41ca-8b64-bbc0753bf815 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:38.916191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e9bee-04d5-41ca-8b64-bbc0753bf815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-692e9bee-04d5-41ca-8b64-bbc0753bf815 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:38.921621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:38.922467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:38.923223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 122/258] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:29:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:38.930419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e9bee-04d5-41ca-8b64-bbc0753bf815 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:38.981132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12218879-ea0d-4eff-9046-c93948333b0b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:39.042936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3509321-9400-4468-8810-354fad21c2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:39.043594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3509321-9400-4468-8810-354fad21c2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:39.043967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3509321-9400-4468-8810-354fad21c2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:39.044242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3509321-9400-4468-8810-354fad21c2c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:39.065472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3509321-9400-4468-8810-354fad21c2c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f3509321-9400-4468-8810-354fad21c2c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:39.075886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:39.076802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:39.077590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:39.077590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:39.077590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:39.077590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:39.077590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:39.077590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:39.077590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:39.077590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:39.077590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:39.077590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:39.077590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:39.077590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:39.078754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 137/259] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:29:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:39.084261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3509321-9400-4468-8810-354fad21c2c4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:39.095085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e9bee-04d5-41ca-8b64-bbc0753bf815 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:39.129889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:39.130995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:39.164988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edf033-082d-4567-913e-5d47ffb49eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:39.165756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edf033-082d-4567-913e-5d47ffb49eda None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:39.166044 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edf033-082d-4567-913e-5d47ffb49eda None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:39.166324 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edf033-082d-4567-913e-5d47ffb49eda None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:39.167004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edf033-082d-4567-913e-5d47ffb49eda None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:39.167361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edf033-082d-4567-913e-5d47ffb49eda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:39.169561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edf033-082d-4567-913e-5d47ffb49eda None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:39.187734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 123/260] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:39.192167 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edf033-082d-4567-913e-5d47ffb49eda None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:39.330411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3509321-9400-4468-8810-354fad21c2c4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:39.414214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edf033-082d-4567-913e-5d47ffb49eda None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:39.427064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f4f00-1146-413f-b8d5-394a7d901a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:39.427737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f4f00-1146-413f-b8d5-394a7d901a37 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:39.428059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f4f00-1146-413f-b8d5-394a7d901a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:39.428334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f4f00-1146-413f-b8d5-394a7d901a37 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:39.429009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f4f00-1146-413f-b8d5-394a7d901a37 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:39.429595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f4f00-1146-413f-b8d5-394a7d901a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:39.433545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f4f00-1146-413f-b8d5-394a7d901a37 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:39.445178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 138/261] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:29:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:39.451040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f4f00-1146-413f-b8d5-394a7d901a37 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:39.490222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8d858-1ae7-45d4-826b-4b17d71c5006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:39.491137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8d858-1ae7-45d4-826b-4b17d71c5006 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:39.491559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8d858-1ae7-45d4-826b-4b17d71c5006 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:39.491970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8d858-1ae7-45d4-826b-4b17d71c5006 None admin] PATH_INFO: `/v3/users/19468ccffc0146e188215f591f4ec198` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:39.493292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8d858-1ae7-45d4-826b-4b17d71c5006 None admin] RBAC: Authorizing `identity:get_user(user_id=19468ccffc0146e188215f591f4ec198)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:39.494929 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8d858-1ae7-45d4-826b-4b17d71c5006 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:39.496694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 124/262] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:39 2022] GET /identity/v3/users/19468ccffc0146e188215f591f4ec198 => generated 283 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:39.505694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8d858-1ae7-45d4-826b-4b17d71c5006 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:39.619384 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:39.633899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:39.651486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f4f00-1146-413f-b8d5-394a7d901a37 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:39.695891 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a76e4-6523-419b-9f91-9f8c963eae00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:39.697232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a76e4-6523-419b-9f91-9f8c963eae00 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:39.697634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a76e4-6523-419b-9f91-9f8c963eae00 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:39.697981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a76e4-6523-419b-9f91-9f8c963eae00 None admin] PATH_INFO: `/v3/users/19468ccffc0146e188215f591f4ec198` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:39.699470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a76e4-6523-419b-9f91-9f8c963eae00 None admin] RBAC: Authorizing `identity:get_user(user_id=19468ccffc0146e188215f591f4ec198)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:39.701559 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a76e4-6523-419b-9f91-9f8c963eae00 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:39.703434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 139/263] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:39 2022] GET /identity/v3/users/19468ccffc0146e188215f591f4ec198 => generated 283 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:39.711707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a76e4-6523-419b-9f91-9f8c963eae00 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:39.743959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8d858-1ae7-45d4-826b-4b17d71c5006 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:39.812275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09556cd5-3316-4eee-99b5-5cee50d21f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:39.812897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09556cd5-3316-4eee-99b5-5cee50d21f62 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:39.813180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09556cd5-3316-4eee-99b5-5cee50d21f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:39.813457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09556cd5-3316-4eee-99b5-5cee50d21f62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:39.930209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a76e4-6523-419b-9f91-9f8c963eae00 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:39.970016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09556cd5-3316-4eee-99b5-5cee50d21f62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:39.971406 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09556cd5-3316-4eee-99b5-5cee50d21f62 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:40.026297 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a96b7-17df-4697-8952-6503f71e6877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:40.026945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a96b7-17df-4697-8952-6503f71e6877 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:40.027231 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a96b7-17df-4697-8952-6503f71e6877 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:40.027510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a96b7-17df-4697-8952-6503f71e6877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:40.057397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:40.074728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:40.150805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a96b7-17df-4697-8952-6503f71e6877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:40.153458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a96b7-17df-4697-8952-6503f71e6877 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:40.159839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 125/264] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:40.165447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09556cd5-3316-4eee-99b5-5cee50d21f62 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:40.334453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09556cd5-3316-4eee-99b5-5cee50d21f62 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:40.357021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 140/265] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:40.362438 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a96b7-17df-4697-8952-6503f71e6877 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:40.406320 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31327cdb-0475-4e64-8552-6dba0f80de74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:40.406953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31327cdb-0475-4e64-8552-6dba0f80de74 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:40.407226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31327cdb-0475-4e64-8552-6dba0f80de74 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:40.407706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31327cdb-0475-4e64-8552-6dba0f80de74 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:40.410101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31327cdb-0475-4e64-8552-6dba0f80de74 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:40.411550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31327cdb-0475-4e64-8552-6dba0f80de74 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:40.414057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 126/266] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:40 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:40.420888 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31327cdb-0475-4e64-8552-6dba0f80de74 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:40.471828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:40.474130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:40.553354 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a96b7-17df-4697-8952-6503f71e6877 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:40.572149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31327cdb-0475-4e64-8552-6dba0f80de74 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:40.606169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a7a3f-dc76-41b1-9431-bf3cfdd62718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:40.606910 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a7a3f-dc76-41b1-9431-bf3cfdd62718 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:40.607196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a7a3f-dc76-41b1-9431-bf3cfdd62718 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:40.607470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a7a3f-dc76-41b1-9431-bf3cfdd62718 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:40.608620 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a7a3f-dc76-41b1-9431-bf3cfdd62718 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:40.610123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a7a3f-dc76-41b1-9431-bf3cfdd62718 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:40.612078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 141/267] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:40 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:40.618091 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a7a3f-dc76-41b1-9431-bf3cfdd62718 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:40.623913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258872b9-a897-4ddb-95b4-c9e5f7821bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:40.624532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258872b9-a897-4ddb-95b4-c9e5f7821bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:40.624811 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258872b9-a897-4ddb-95b4-c9e5f7821bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:40.625081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258872b9-a897-4ddb-95b4-c9e5f7821bf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:40.625875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258872b9-a897-4ddb-95b4-c9e5f7821bf7 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85, user.id=19468ccffc0146e188215f591f4ec198) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:40.626265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258872b9-a897-4ddb-95b4-c9e5f7821bf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:40.627780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258872b9-a897-4ddb-95b4-c9e5f7821bf7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:40.635033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 127/268] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:40 2022] GET /identity/v3/role_assignments?user.id=19468ccffc0146e188215f591f4ec198&scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:40.753450 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a7a3f-dc76-41b1-9431-bf3cfdd62718 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:40.755644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec9569-c7f3-4473-9fc1-ab88680952ca None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:40.756130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec9569-c7f3-4473-9fc1-ab88680952ca None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:40.756596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec9569-c7f3-4473-9fc1-ab88680952ca None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:40.758469 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 128/269] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:40.764239 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b12189-0514-4370-aac7-732b484f31d3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:40.764885 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b12189-0514-4370-aac7-732b484f31d3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:40.765364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b12189-0514-4370-aac7-732b484f31d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:40.780006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:40.787342 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-33b12189-0514-4370-aac7-732b484f31d3 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:40.788874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-33b12189-0514-4370-aac7-732b484f31d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:40.791575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cbc0a-3e3e-43d4-b9e7-348143aed932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:40.792250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cbc0a-3e3e-43d4-b9e7-348143aed932 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:40.792486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cbc0a-3e3e-43d4-b9e7-348143aed932 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:40.792761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cbc0a-3e3e-43d4-b9e7-348143aed932 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:40.793579 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40cbc0a-3e3e-43d4-b9e7-348143aed932 None admin] RBAC: Adding query filter params (role.id=1d1b5b96118040f9a4fd019bbcd93071, scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85, user.id=19468ccffc0146e188215f591f4ec198) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:40.793843 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40cbc0a-3e3e-43d4-b9e7-348143aed932 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:40.795293 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40cbc0a-3e3e-43d4-b9e7-348143aed932 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:40.801854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 142/270] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:40 2022] GET /identity/v3/role_assignments?user.id=19468ccffc0146e188215f591f4ec198&scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85&role.id=1d1b5b96118040f9a4fd019bbcd93071 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:40.880957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 129/271] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:40.919624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cbc0a-3e3e-43d4-b9e7-348143aed932 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:41.035021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cbc0a-3e3e-43d4-b9e7-348143aed932 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:41.070553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8508d08-7ada-457c-afb0-631949f97850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:41.071200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8508d08-7ada-457c-afb0-631949f97850 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:41.071481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8508d08-7ada-457c-afb0-631949f97850 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:41.071784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8508d08-7ada-457c-afb0-631949f97850 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:41.082581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8508d08-7ada-457c-afb0-631949f97850 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-e8508d08-7ada-457c-afb0-631949f97850 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:41.088425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:41.089308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:41.090104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 143/272] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:41.097089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b12189-0514-4370-aac7-732b484f31d3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:41.110358 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:41.208476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b12189-0514-4370-aac7-732b484f31d3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:41.255461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3d7fc-043d-40fa-b621-b111c6af17f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:41.256082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3d7fc-043d-40fa-b621-b111c6af17f7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:41.256358 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3d7fc-043d-40fa-b621-b111c6af17f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:41.256631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3d7fc-043d-40fa-b621-b111c6af17f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:41.257305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3d7fc-043d-40fa-b621-b111c6af17f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:41.257634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3d7fc-043d-40fa-b621-b111c6af17f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:41.259524 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3d7fc-043d-40fa-b621-b111c6af17f7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:41.268970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 130/273] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:41.275419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8508d08-7ada-457c-afb0-631949f97850 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:41.309810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8925-035a-4d66-8cc2-5e0b150e32e1 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:41.310108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8925-035a-4d66-8cc2-5e0b150e32e1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:41.310360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8925-035a-4d66-8cc2-5e0b150e32e1 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:41.312074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 131/274] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:41.320132 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa012fe-937b-4175-a731-5df3372494e6 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:41.320418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa012fe-937b-4175-a731-5df3372494e6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:41.320688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa012fe-937b-4175-a731-5df3372494e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:41.338504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:41.343212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6aa012fe-937b-4175-a731-5df3372494e6 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:41.344411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6aa012fe-937b-4175-a731-5df3372494e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:41.410294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8508d08-7ada-457c-afb0-631949f97850 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:41.430585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 132/275] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:41.451345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11144be-0218-49d1-bca8-9406fa3971ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:41.451992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11144be-0218-49d1-bca8-9406fa3971ad None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:41.452276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11144be-0218-49d1-bca8-9406fa3971ad None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:41.452551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11144be-0218-49d1-bca8-9406fa3971ad None admin] PATH_INFO: `/v3/users/5a6d1e56e8184121b5aba03ff3ef567e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:41.456174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa012fe-937b-4175-a731-5df3372494e6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:41.458375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:41.480586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11144be-0218-49d1-bca8-9406fa3971ad None admin] RBAC: Authorizing `identity:get_user(user_id=5a6d1e56e8184121b5aba03ff3ef567e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:41.482029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11144be-0218-49d1-bca8-9406fa3971ad None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:41.484072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 144/276] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:41 2022] GET /identity/v3/users/5a6d1e56e8184121b5aba03ff3ef567e => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:41.489583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11144be-0218-49d1-bca8-9406fa3971ad None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:41.588591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa012fe-937b-4175-a731-5df3372494e6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:41.605927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11144be-0218-49d1-bca8-9406fa3971ad None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:41.607567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:41.625158 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564b5c7-c66a-4802-a008-3155c4c801e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:41.625829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564b5c7-c66a-4802-a008-3155c4c801e0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:41.626110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564b5c7-c66a-4802-a008-3155c4c801e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:41.626385 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564b5c7-c66a-4802-a008-3155c4c801e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:41.627507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564b5c7-c66a-4802-a008-3155c4c801e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:41.628946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564b5c7-c66a-4802-a008-3155c4c801e0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:41.635201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 133/277] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:41.635201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bf6ac-ec49-4772-8514-cda09a9cae49 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:41.635201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bf6ac-ec49-4772-8514-cda09a9cae49 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:41.635201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bf6ac-ec49-4772-8514-cda09a9cae49 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:41.635201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 134/278] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:41.638165 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bf6ac-ec49-4772-8514-cda09a9cae49 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:41.647104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3937e-9772-4fa4-a529-645f5bc96ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:41.647715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3937e-9772-4fa4-a529-645f5bc96ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:41.648002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3937e-9772-4fa4-a529-645f5bc96ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:41.648272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3937e-9772-4fa4-a529-645f5bc96ff8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:41.648996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3937e-9772-4fa4-a529-645f5bc96ff8 None admin] RBAC: Adding query filter params (role.id=08a7c861d84b43d7a4da431f31f3c2d7, user.id=5a6d1e56e8184121b5aba03ff3ef567e, scope.system=all) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:41.649302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3937e-9772-4fa4-a529-645f5bc96ff8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:41.651274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3937e-9772-4fa4-a529-645f5bc96ff8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:41.656179 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 145/279] 10.222.0.57 () {56 vars in 1308 bytes} [Tue Aug 9 18:29:41 2022] GET /identity/v3/role_assignments?user.id=5a6d1e56e8184121b5aba03ff3ef567e&scope.system=all&role.id=08a7c861d84b43d7a4da431f31f3c2d7 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:41.664282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62625-c9c0-49e4-b85e-bc104a287437 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:41.664282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62625-c9c0-49e4-b85e-bc104a287437 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:41.664282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62625-c9c0-49e4-b85e-bc104a287437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:41.677703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-98d62625-c9c0-49e4-b85e-bc104a287437 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:41.679041 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-98d62625-c9c0-49e4-b85e-bc104a287437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:41.731896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:41.761617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 146/280] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:41.776423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bf6ac-ec49-4772-8514-cda09a9cae49 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:41.778457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9a04e-5e24-47a3-a4cf-669cbb086c6b None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:41.778810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9a04e-5e24-47a3-a4cf-669cbb086c6b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:41.779152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9a04e-5e24-47a3-a4cf-669cbb086c6b None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:41.780820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 147/281] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:29:41.786835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43436482-c88d-4631-8c80-84e379ebe4ab None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:41.787111 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43436482-c88d-4631-8c80-84e379ebe4ab None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:41.787383 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43436482-c88d-4631-8c80-84e379ebe4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:41.807426 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-43436482-c88d-4631-8c80-84e379ebe4ab None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:41.808672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-43436482-c88d-4631-8c80-84e379ebe4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:41.816875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b09d8d-4313-4b80-8e1d-d29df58cd22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:41.817739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b09d8d-4313-4b80-8e1d-d29df58cd22b None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:41.817873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b09d8d-4313-4b80-8e1d-d29df58cd22b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:41.818053 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b09d8d-4313-4b80-8e1d-d29df58cd22b None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:41.818702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b09d8d-4313-4b80-8e1d-d29df58cd22b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:41.820153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b09d8d-4313-4b80-8e1d-d29df58cd22b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:41.886232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 148/282] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:41.894124 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43436482-c88d-4631-8c80-84e379ebe4ab None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:41.914894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:41.936661 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 135/283] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:29:41 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:29:41.937406 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b09d8d-4313-4b80-8e1d-d29df58cd22b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:42.013987 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43436482-c88d-4631-8c80-84e379ebe4ab None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:42.058817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3bee8-9f22-40c6-a9a5-2b6cacf50e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:42.059786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3bee8-9f22-40c6-a9a5-2b6cacf50e4e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:42.060227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3bee8-9f22-40c6-a9a5-2b6cacf50e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:42.060631 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3bee8-9f22-40c6-a9a5-2b6cacf50e4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:42.075512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3bee8-9f22-40c6-a9a5-2b6cacf50e4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-66c3bee8-9f22-40c6-a9a5-2b6cacf50e4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:42.080777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:42.083859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:42.084644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 149/284] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:42.088452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3bee8-9f22-40c6-a9a5-2b6cacf50e4e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:42.094932 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b09d8d-4313-4b80-8e1d-d29df58cd22b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:42.128001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:42.136960 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0adbf2-eb7a-4a93-9110-a79130d041b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:42.137691 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0adbf2-eb7a-4a93-9110-a79130d041b8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:42.137975 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0adbf2-eb7a-4a93-9110-a79130d041b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:42.138249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0adbf2-eb7a-4a93-9110-a79130d041b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:42.156185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0adbf2-eb7a-4a93-9110-a79130d041b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-fb0adbf2-eb7a-4a93-9110-a79130d041b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:42.161286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:42.162296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:42.163278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 136/285] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:42.168488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0adbf2-eb7a-4a93-9110-a79130d041b8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:42.200300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:42.219653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3bee8-9f22-40c6-a9a5-2b6cacf50e4e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:42.257372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c0e3c-18d3-4426-92b2-550d8769bf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:42.258017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c0e3c-18d3-4426-92b2-550d8769bf70 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:42.258354 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c0e3c-18d3-4426-92b2-550d8769bf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:42.258725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c0e3c-18d3-4426-92b2-550d8769bf70 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:42.259935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c0e3c-18d3-4426-92b2-550d8769bf70 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:42.260237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c0e3c-18d3-4426-92b2-550d8769bf70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:42.261747 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c0e3c-18d3-4426-92b2-550d8769bf70 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:42.273883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 150/286] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:42.278635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c0e3c-18d3-4426-92b2-550d8769bf70 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:42.318060 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0adbf2-eb7a-4a93-9110-a79130d041b8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:42.364318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f7d76-60ea-4d08-98ef-778e952cacb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:42.365283 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f7d76-60ea-4d08-98ef-778e952cacb0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:42.365828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f7d76-60ea-4d08-98ef-778e952cacb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:42.366315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f7d76-60ea-4d08-98ef-778e952cacb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:42.367182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f7d76-60ea-4d08-98ef-778e952cacb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:42.367664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f7d76-60ea-4d08-98ef-778e952cacb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:42.369388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f7d76-60ea-4d08-98ef-778e952cacb0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:42.381603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 137/287] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:42.386922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f7d76-60ea-4d08-98ef-778e952cacb0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:42.406865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c0e3c-18d3-4426-92b2-550d8769bf70 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:42.440353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:42.446437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69029f-b341-46c8-a76e-4ee2fc3ebf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:42.447437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69029f-b341-46c8-a76e-4ee2fc3ebf8d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:42.447947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69029f-b341-46c8-a76e-4ee2fc3ebf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:42.448431 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69029f-b341-46c8-a76e-4ee2fc3ebf8d None admin] PATH_INFO: `/v3/users/dc810d059ad14efe91729d2b7913a166` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:42.449827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69029f-b341-46c8-a76e-4ee2fc3ebf8d None admin] RBAC: Authorizing `identity:get_user(user_id=dc810d059ad14efe91729d2b7913a166)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:42.451762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69029f-b341-46c8-a76e-4ee2fc3ebf8d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:42.453657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 151/288] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:42 2022] GET /identity/v3/users/dc810d059ad14efe91729d2b7913a166 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:42.459607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69029f-b341-46c8-a76e-4ee2fc3ebf8d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:42.480927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:42.496910 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f7d76-60ea-4d08-98ef-778e952cacb0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:42.534723 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3c12e-255b-4fc0-bb21-72e9578b9254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:42.535269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3c12e-255b-4fc0-bb21-72e9578b9254 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:42.535552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3c12e-255b-4fc0-bb21-72e9578b9254 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:42.535825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3c12e-255b-4fc0-bb21-72e9578b9254 None admin] PATH_INFO: `/v3/users/34dbeffb7d6c4574bcfed5188e835175` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:42.537334 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3c12e-255b-4fc0-bb21-72e9578b9254 None admin] RBAC: Authorizing `identity:get_user(user_id=34dbeffb7d6c4574bcfed5188e835175)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:42.539129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3c12e-255b-4fc0-bb21-72e9578b9254 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:42.545563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 138/289] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:42 2022] GET /identity/v3/users/34dbeffb7d6c4574bcfed5188e835175 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:42.547771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3c12e-255b-4fc0-bb21-72e9578b9254 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:42.674648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3c12e-255b-4fc0-bb21-72e9578b9254 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:42.675418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69029f-b341-46c8-a76e-4ee2fc3ebf8d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:42.710079 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:42.714847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4a8c1-37d3-45b1-998d-1a929f442593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:42.715654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4a8c1-37d3-45b1-998d-1a929f442593 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:42.715943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4a8c1-37d3-45b1-998d-1a929f442593 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:42.716219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4a8c1-37d3-45b1-998d-1a929f442593 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:42.757954 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b067248-3b8c-492d-9a00-093a7df81be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:42.758600 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b067248-3b8c-492d-9a00-093a7df81be6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:42.758883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b067248-3b8c-492d-9a00-093a7df81be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:42.759695 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b067248-3b8c-492d-9a00-093a7df81be6 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:42.760867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b067248-3b8c-492d-9a00-093a7df81be6 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:42.762371 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b067248-3b8c-492d-9a00-093a7df81be6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:42.764118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 152/290] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:42 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:42.770048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b067248-3b8c-492d-9a00-093a7df81be6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:42.796938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4a8c1-37d3-45b1-998d-1a929f442593 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:42.798598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4a8c1-37d3-45b1-998d-1a929f442593 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:42.921085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 139/291] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:42.927129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4a8c1-37d3-45b1-998d-1a929f442593 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:42.950056 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:42.952787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:42.991665 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b067248-3b8c-492d-9a00-093a7df81be6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:43.044300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4a8c1-37d3-45b1-998d-1a929f442593 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:43.082692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b159c3-e9e7-4c13-aa4b-1dac25b07aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:43.083374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b159c3-e9e7-4c13-aa4b-1dac25b07aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:43.083661 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b159c3-e9e7-4c13-aa4b-1dac25b07aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:43.083935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b159c3-e9e7-4c13-aa4b-1dac25b07aa3 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:43.085074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b159c3-e9e7-4c13-aa4b-1dac25b07aa3 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:43.086727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b159c3-e9e7-4c13-aa4b-1dac25b07aa3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:43.088461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 140/292] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:42 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:43.095068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b159c3-e9e7-4c13-aa4b-1dac25b07aa3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:43.100775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c790c-d2e5-49d0-a111-487d99523695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:43.103373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c790c-d2e5-49d0-a111-487d99523695 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:43.103373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c790c-d2e5-49d0-a111-487d99523695 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:43.103373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c790c-d2e5-49d0-a111-487d99523695 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85/users/dc810d059ad14efe91729d2b7913a166/roles/1b625e8349d04de08806337ee4bd9478` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:43.104736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c790c-d2e5-49d0-a111-487d99523695 None admin] RBAC: Authorizing `identity:create_grant(project_id=38ed82c9c6da41e596c28a3fb28c8c85, user_id=dc810d059ad14efe91729d2b7913a166, role_id=1b625e8349d04de08806337ee4bd9478)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:43.106233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c790c-d2e5-49d0-a111-487d99523695 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:43.188039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 153/293] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:29:42 2022] PUT /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85/users/dc810d059ad14efe91729d2b7913a166/roles/1b625e8349d04de08806337ee4bd9478 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:29:43.207301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:43.228192 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b159c3-e9e7-4c13-aa4b-1dac25b07aa3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:43.276830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a00207-5724-4aed-8d03-e5db61f3fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:43.277507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a00207-5724-4aed-8d03-e5db61f3fd96 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:43.277825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a00207-5724-4aed-8d03-e5db61f3fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:43.278106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a00207-5724-4aed-8d03-e5db61f3fd96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:43.278874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a00207-5724-4aed-8d03-e5db61f3fd96 None admin] RBAC: Adding query filter params (role.id=08a7c861d84b43d7a4da431f31f3c2d7, scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85, user.id=34dbeffb7d6c4574bcfed5188e835175) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:43.279188 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a00207-5724-4aed-8d03-e5db61f3fd96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:43.280881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a00207-5724-4aed-8d03-e5db61f3fd96 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:43.291183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 141/294] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:43 2022] GET /identity/v3/role_assignments?user.id=34dbeffb7d6c4574bcfed5188e835175&scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85&role.id=08a7c861d84b43d7a4da431f31f3c2d7 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:43.891653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283349a-4808-4845-8849-f1e63b16c0eb None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:43.892159 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283349a-4808-4845-8849-f1e63b16c0eb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:43.892261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283349a-4808-4845-8849-f1e63b16c0eb None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:43.896044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 154/295] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:43.902306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3436b-534d-421c-b316-37639cf2e387 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:43.902754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3436b-534d-421c-b316-37639cf2e387 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:43.902942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3436b-534d-421c-b316-37639cf2e387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:43.921502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-71a3436b-534d-421c-b316-37639cf2e387 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:43.922792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-71a3436b-534d-421c-b316-37639cf2e387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:43.991594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 142/296] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:44.013815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283349a-4808-4845-8849-f1e63b16c0eb None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:44.123238 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283349a-4808-4845-8849-f1e63b16c0eb None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:44.159212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afae36-7249-4f86-b66d-ea67c9a1e48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:44.159773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afae36-7249-4f86-b66d-ea67c9a1e48f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:44.160046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afae36-7249-4f86-b66d-ea67c9a1e48f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:44.160314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afae36-7249-4f86-b66d-ea67c9a1e48f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:44.176580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00afae36-7249-4f86-b66d-ea67c9a1e48f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-00afae36-7249-4f86-b66d-ea67c9a1e48f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:44.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:44.182818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:44.183603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 155/297] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:44.188826 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3436b-534d-421c-b316-37639cf2e387 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:44.278582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:44.295690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3436b-534d-421c-b316-37639cf2e387 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:44.331778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc837d-f463-46b9-8ce3-5e2910042a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:44.332385 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc837d-f463-46b9-8ce3-5e2910042a83 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:44.332663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc837d-f463-46b9-8ce3-5e2910042a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:44.332938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc837d-f463-46b9-8ce3-5e2910042a83 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:44.333635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc837d-f463-46b9-8ce3-5e2910042a83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:44.333942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc837d-f463-46b9-8ce3-5e2910042a83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:44.335379 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc837d-f463-46b9-8ce3-5e2910042a83 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:44.344587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 143/298] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:44.350284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afae36-7249-4f86-b66d-ea67c9a1e48f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:44.352381 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab218e5-98c1-45c8-99a9-9cc7e0d93d92 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:44.352911 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab218e5-98c1-45c8-99a9-9cc7e0d93d92 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:44.353386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab218e5-98c1-45c8-99a9-9cc7e0d93d92 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:44.355004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:44.355400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 144/299] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:29:44.360018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb19b63-5632-46fb-bf4d-38290c6ac520 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:44.360302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb19b63-5632-46fb-bf4d-38290c6ac520 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:44.360573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb19b63-5632-46fb-bf4d-38290c6ac520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:44.381079 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-afb19b63-5632-46fb-bf4d-38290c6ac520 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:44.382911 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-afb19b63-5632-46fb-bf4d-38290c6ac520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:44.461029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afae36-7249-4f86-b66d-ea67c9a1e48f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:44.469176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 145/300] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:44.492175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb19b63-5632-46fb-bf4d-38290c6ac520 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:44.501077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80706ae4-78f3-46d1-9ca1-001c06828073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:44.502228 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80706ae4-78f3-46d1-9ca1-001c06828073 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:44.502783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80706ae4-78f3-46d1-9ca1-001c06828073 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:44.505634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80706ae4-78f3-46d1-9ca1-001c06828073 None admin] PATH_INFO: `/v3/users/5a6d1e56e8184121b5aba03ff3ef567e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:44.505634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80706ae4-78f3-46d1-9ca1-001c06828073 None admin] RBAC: Authorizing `identity:get_user(user_id=5a6d1e56e8184121b5aba03ff3ef567e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:44.506319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80706ae4-78f3-46d1-9ca1-001c06828073 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:44.508131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 156/301] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:44 2022] GET /identity/v3/users/5a6d1e56e8184121b5aba03ff3ef567e => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:44.514008 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80706ae4-78f3-46d1-9ca1-001c06828073 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:44.579748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:44.616482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb19b63-5632-46fb-bf4d-38290c6ac520 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:44.617944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:44.655679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97bc55-6e04-4dab-a089-ffa6a2d6f445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:44.656414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97bc55-6e04-4dab-a089-ffa6a2d6f445 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:44.656696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97bc55-6e04-4dab-a089-ffa6a2d6f445 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:44.657028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97bc55-6e04-4dab-a089-ffa6a2d6f445 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:44.658787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80706ae4-78f3-46d1-9ca1-001c06828073 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:44.668856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97bc55-6e04-4dab-a089-ffa6a2d6f445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-4f97bc55-6e04-4dab-a089-ffa6a2d6f445 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:44.674913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:44.675786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:44.676647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 146/302] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:44.683296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162ff84-be73-4e55-8a1d-d5cf78a322f9 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:44.683580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162ff84-be73-4e55-8a1d-d5cf78a322f9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:44.683852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162ff84-be73-4e55-8a1d-d5cf78a322f9 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:44.685397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 147/303] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:44.687548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162ff84-be73-4e55-8a1d-d5cf78a322f9 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:44.697342 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7cdc0-c7eb-4e32-ae55-764ba99d9c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:44.698027 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7cdc0-c7eb-4e32-ae55-764ba99d9c1b None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:44.698369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7cdc0-c7eb-4e32-ae55-764ba99d9c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:44.698650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7cdc0-c7eb-4e32-ae55-764ba99d9c1b None admin] PATH_INFO: `/v3/system/users/5a6d1e56e8184121b5aba03ff3ef567e/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:44.700057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7cdc0-c7eb-4e32-ae55-764ba99d9c1b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5a6d1e56e8184121b5aba03ff3ef567e, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:44.701497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7cdc0-c7eb-4e32-ae55-764ba99d9c1b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:44.789636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 157/304] 10.222.0.57 () {58 vars in 1334 bytes} [Tue Aug 9 18:29:44 2022] PUT /identity/v3/system/users/5a6d1e56e8184121b5aba03ff3ef567e/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:29:44.789636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145ef42-5a63-429b-aea4-9c7782925490 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:44.789636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145ef42-5a63-429b-aea4-9c7782925490 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:44.789636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145ef42-5a63-429b-aea4-9c7782925490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:44.802141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162ff84-be73-4e55-8a1d-d5cf78a322f9 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:44.803914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f145ef42-5a63-429b-aea4-9c7782925490 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:44.805420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f145ef42-5a63-429b-aea4-9c7782925490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:44.844519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:44.852527 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30810e23-ba22-45e4-8993-c1a439f8364a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:44.853599 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30810e23-ba22-45e4-8993-c1a439f8364a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:44.854138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30810e23-ba22-45e4-8993-c1a439f8364a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:44.854614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30810e23-ba22-45e4-8993-c1a439f8364a None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:44.856355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30810e23-ba22-45e4-8993-c1a439f8364a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:44.856899 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30810e23-ba22-45e4-8993-c1a439f8364a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:44.859003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30810e23-ba22-45e4-8993-c1a439f8364a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:44.872359 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 148/305] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:44.877473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30810e23-ba22-45e4-8993-c1a439f8364a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:44.891921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 158/306] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:44.918147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145ef42-5a63-429b-aea4-9c7782925490 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:45.004070 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30810e23-ba22-45e4-8993-c1a439f8364a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:45.006665 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:45.045101 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145ef42-5a63-429b-aea4-9c7782925490 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:45.049331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284546d-401f-46bd-a39c-a1e5aeeee5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:45.050050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284546d-401f-46bd-a39c-a1e5aeeee5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:45.050332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284546d-401f-46bd-a39c-a1e5aeeee5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:45.050609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284546d-401f-46bd-a39c-a1e5aeeee5b5 None admin] PATH_INFO: `/v3/users/e0d66d329fec44d2b1d41e8363db9fbf` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:45.067305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284546d-401f-46bd-a39c-a1e5aeeee5b5 None admin] RBAC: Authorizing `identity:get_user(user_id=e0d66d329fec44d2b1d41e8363db9fbf)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:45.070858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284546d-401f-46bd-a39c-a1e5aeeee5b5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:45.072293 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 149/307] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:44 2022] GET /identity/v3/users/e0d66d329fec44d2b1d41e8363db9fbf => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:45.078227 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284546d-401f-46bd-a39c-a1e5aeeee5b5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:45.093813 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a191c2f-8fcd-4280-8102-568ede10737e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:45.094404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a191c2f-8fcd-4280-8102-568ede10737e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:45.094791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a191c2f-8fcd-4280-8102-568ede10737e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:45.095098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a191c2f-8fcd-4280-8102-568ede10737e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:45.096219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a191c2f-8fcd-4280-8102-568ede10737e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:45.097863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a191c2f-8fcd-4280-8102-568ede10737e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:45.099312 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 159/308] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:45.106032 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a191c2f-8fcd-4280-8102-568ede10737e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:45.150531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:45.213161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284546d-401f-46bd-a39c-a1e5aeeee5b5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:45.222772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a191c2f-8fcd-4280-8102-568ede10737e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:45.254521 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffff523-f672-4247-9d5b-491afa7dac67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:45.255136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffff523-f672-4247-9d5b-491afa7dac67 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:45.255415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffff523-f672-4247-9d5b-491afa7dac67 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:45.255686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffff523-f672-4247-9d5b-491afa7dac67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:45.256683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffff523-f672-4247-9d5b-491afa7dac67 None admin] RBAC: Adding query filter params (role.id=1b625e8349d04de08806337ee4bd9478, user.id=e0d66d329fec44d2b1d41e8363db9fbf, scope.system=all) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:45.257407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffff523-f672-4247-9d5b-491afa7dac67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:45.259662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffff523-f672-4247-9d5b-491afa7dac67 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:45.277087 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 150/309] 10.222.0.57 () {56 vars in 1308 bytes} [Tue Aug 9 18:29:45 2022] GET /identity/v3/role_assignments?user.id=e0d66d329fec44d2b1d41e8363db9fbf&scope.system=all&role.id=1b625e8349d04de08806337ee4bd9478 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:45.279258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a2828-8f64-4b6e-8f1b-ced3a836645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:45.279895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a2828-8f64-4b6e-8f1b-ced3a836645a None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:45.280174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a2828-8f64-4b6e-8f1b-ced3a836645a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:45.280445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a2828-8f64-4b6e-8f1b-ced3a836645a None admin] PATH_INFO: `/v3/groups` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:45.281092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a2828-8f64-4b6e-8f1b-ced3a836645a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:45.282611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a2828-8f64-4b6e-8f1b-ced3a836645a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:45.287309 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:45.354406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 160/310] 10.222.0.57 () {60 vars in 1134 bytes} [Tue Aug 9 18:29:45 2022] POST /identity/v3/groups => generated 229 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:29:45.514412 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4db32-2a45-49aa-bfb9-4173e54bc935 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:45.514781 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4db32-2a45-49aa-bfb9-4173e54bc935 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:45.515004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4db32-2a45-49aa-bfb9-4173e54bc935 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:45.517040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 151/311] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:45.523260 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ad65c-2957-4262-97e3-a978b5425727 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:45.523865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ad65c-2957-4262-97e3-a978b5425727 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:45.524343 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ad65c-2957-4262-97e3-a978b5425727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:45.544326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4f0ad65c-2957-4262-97e3-a978b5425727 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:45.545925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4f0ad65c-2957-4262-97e3-a978b5425727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:45.621600 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 161/312] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:45.652890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4db32-2a45-49aa-bfb9-4173e54bc935 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:45.759762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4db32-2a45-49aa-bfb9-4173e54bc935 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:45.777924 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:45.797810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f99812-f9dc-4a74-a50d-401f9f68fc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:45.798481 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f99812-f9dc-4a74-a50d-401f9f68fc15 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:45.798764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f99812-f9dc-4a74-a50d-401f9f68fc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:45.799030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f99812-f9dc-4a74-a50d-401f9f68fc15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:45.810665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f99812-f9dc-4a74-a50d-401f9f68fc15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-24f99812-f9dc-4a74-a50d-401f9f68fc15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:45.815946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:45.816882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:45.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:45.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:45.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:45.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:45.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:45.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:45.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:45.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:45.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:45.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:45.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:45.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:45.821940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 152/313] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:45.824746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ad65c-2957-4262-97e3-a978b5425727 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:45.929109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ad65c-2957-4262-97e3-a978b5425727 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:45.957440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:45.968593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d44f02-3bb4-4351-8a49-066e2bcf1f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:45.969217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d44f02-3bb4-4351-8a49-066e2bcf1f8e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:45.981937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d44f02-3bb4-4351-8a49-066e2bcf1f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:45.981937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d44f02-3bb4-4351-8a49-066e2bcf1f8e None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:45.981937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d44f02-3bb4-4351-8a49-066e2bcf1f8e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:45.981937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d44f02-3bb4-4351-8a49-066e2bcf1f8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:45.981937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d44f02-3bb4-4351-8a49-066e2bcf1f8e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:45.981937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 162/314] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:45.987617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f99812-f9dc-4a74-a50d-401f9f68fc15 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:46.098069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f99812-f9dc-4a74-a50d-401f9f68fc15 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:46.136559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1114a-70bb-46e3-8534-03d1697937a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:46.137186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1114a-70bb-46e3-8534-03d1697937a5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:46.156155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1114a-70bb-46e3-8534-03d1697937a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:46.156155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1114a-70bb-46e3-8534-03d1697937a5 None admin] PATH_INFO: `/v3/users/dc810d059ad14efe91729d2b7913a166` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:46.156155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1114a-70bb-46e3-8534-03d1697937a5 None admin] RBAC: Authorizing `identity:get_user(user_id=dc810d059ad14efe91729d2b7913a166)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:46.156155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1114a-70bb-46e3-8534-03d1697937a5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:46.156155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 153/315] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:45 2022] GET /identity/v3/users/dc810d059ad14efe91729d2b7913a166 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:46.156503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d44f02-3bb4-4351-8a49-066e2bcf1f8e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:46.253439 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d44f02-3bb4-4351-8a49-066e2bcf1f8e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:46.298650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224f3a9-36f3-4dd5-815f-1c9aa805188b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:46.299255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224f3a9-36f3-4dd5-815f-1c9aa805188b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:46.299531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224f3a9-36f3-4dd5-815f-1c9aa805188b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:46.299798 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224f3a9-36f3-4dd5-815f-1c9aa805188b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:46.357410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:46.371630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224f3a9-36f3-4dd5-815f-1c9aa805188b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:46.373138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224f3a9-36f3-4dd5-815f-1c9aa805188b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:46.474754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 163/316] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:46.480306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1114a-70bb-46e3-8534-03d1697937a5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:46.529015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:46.588716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1114a-70bb-46e3-8534-03d1697937a5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:46.625557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcaa64-5359-49e7-ad17-6a37a5ee8843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:46.626195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcaa64-5359-49e7-ad17-6a37a5ee8843 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:46.626477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcaa64-5359-49e7-ad17-6a37a5ee8843 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:46.626751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcaa64-5359-49e7-ad17-6a37a5ee8843 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:46.627886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcaa64-5359-49e7-ad17-6a37a5ee8843 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:46.629340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcaa64-5359-49e7-ad17-6a37a5ee8843 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:46.631392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 154/317] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:46 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:46.637448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224f3a9-36f3-4dd5-815f-1c9aa805188b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:46.725929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:46.740295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224f3a9-36f3-4dd5-815f-1c9aa805188b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:46.775784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31d5c2-4607-43a9-8969-3ef4063aa5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:46.776441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31d5c2-4607-43a9-8969-3ef4063aa5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:46.776720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31d5c2-4607-43a9-8969-3ef4063aa5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:46.776989 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31d5c2-4607-43a9-8969-3ef4063aa5d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:46.777742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31d5c2-4607-43a9-8969-3ef4063aa5d1 None admin] RBAC: Adding query filter params (role.id=1b625e8349d04de08806337ee4bd9478, scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85, user.id=dc810d059ad14efe91729d2b7913a166) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:46.778045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31d5c2-4607-43a9-8969-3ef4063aa5d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:46.779574 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31d5c2-4607-43a9-8969-3ef4063aa5d1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:46.785668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 164/318] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:29:46 2022] GET /identity/v3/role_assignments?user.id=dc810d059ad14efe91729d2b7913a166&scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85&role.id=1b625e8349d04de08806337ee4bd9478 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:46.908463 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971c796-bf6a-4cb0-9eca-eafa74a5b473 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:46.908811 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971c796-bf6a-4cb0-9eca-eafa74a5b473 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:46.909016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971c796-bf6a-4cb0-9eca-eafa74a5b473 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:46.910666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 155/319] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:46.916897 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba5d15-c086-456b-ab76-323a1136bdc9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:46.917506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba5d15-c086-456b-ab76-323a1136bdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:46.917991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba5d15-c086-456b-ab76-323a1136bdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:46.937353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-72ba5d15-c086-456b-ab76-323a1136bdc9 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:46.939327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-72ba5d15-c086-456b-ab76-323a1136bdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:47.013627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 165/320] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:47.034224 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971c796-bf6a-4cb0-9eca-eafa74a5b473 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:47.136725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971c796-bf6a-4cb0-9eca-eafa74a5b473 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:47.147579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:47.171772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e48a6-7b56-4e6f-8a33-241c93a01c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:47.172400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e48a6-7b56-4e6f-8a33-241c93a01c42 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:47.172681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e48a6-7b56-4e6f-8a33-241c93a01c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:47.172995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e48a6-7b56-4e6f-8a33-241c93a01c42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:47.183882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e48a6-7b56-4e6f-8a33-241c93a01c42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-4b5e48a6-7b56-4e6f-8a33-241c93a01c42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:47.189110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:47.190235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:47.191037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:47.191037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:47.191037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:47.191037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:47.191037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:47.191037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:47.191037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:47.191037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:47.191037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:47.191037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:47.191037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:47.191037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:47.193350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 156/321] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:47.203219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba5d15-c086-456b-ab76-323a1136bdc9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:47.307700 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:47.327099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba5d15-c086-456b-ab76-323a1136bdc9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:47.361167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27905387-a1c3-42e8-9bb1-2630d218fec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:47.361805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27905387-a1c3-42e8-9bb1-2630d218fec3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:47.362135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27905387-a1c3-42e8-9bb1-2630d218fec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:47.362408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27905387-a1c3-42e8-9bb1-2630d218fec3 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:47.363065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27905387-a1c3-42e8-9bb1-2630d218fec3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:47.363367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27905387-a1c3-42e8-9bb1-2630d218fec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:47.364731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27905387-a1c3-42e8-9bb1-2630d218fec3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:47.378137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 166/322] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:47.379552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e48a6-7b56-4e6f-8a33-241c93a01c42 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:47.478485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250599f-491b-43b7-8f09-c16d78e35f7e None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:47.478845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250599f-491b-43b7-8f09-c16d78e35f7e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:47.479114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250599f-491b-43b7-8f09-c16d78e35f7e None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:47.480540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 167/323] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:47 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:47.485845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc06b3-ef81-4c31-ac41-95d1e9736bd5 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:47.486125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc06b3-ef81-4c31-ac41-95d1e9736bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:47.486395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc06b3-ef81-4c31-ac41-95d1e9736bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:47.486573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e48a6-7b56-4e6f-8a33-241c93a01c42 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:47.504202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-10bc06b3-ef81-4c31-ac41-95d1e9736bd5 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:47.505361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-10bc06b3-ef81-4c31-ac41-95d1e9736bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:47.524159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ffe7c-d727-4c54-afde-b6147a71a6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:47.525155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ffe7c-d727-4c54-afde-b6147a71a6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:47.525448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ffe7c-d727-4c54-afde-b6147a71a6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:47.525785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ffe7c-d727-4c54-afde-b6147a71a6c8 None admin] PATH_INFO: `/v3/users/5a6d1e56e8184121b5aba03ff3ef567e` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:47.526917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ffe7c-d727-4c54-afde-b6147a71a6c8 None admin] RBAC: Authorizing `identity:get_user(user_id=5a6d1e56e8184121b5aba03ff3ef567e)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:47.528363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ffe7c-d727-4c54-afde-b6147a71a6c8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:47.530173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 157/324] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:47 2022] GET /identity/v3/users/5a6d1e56e8184121b5aba03ff3ef567e => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:47.535688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ffe7c-d727-4c54-afde-b6147a71a6c8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:47.584245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 168/325] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:47.585236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:47.585411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5f006-2971-40d6-a0f1-ae784ee12e53 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:47.585728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5f006-2971-40d6-a0f1-ae784ee12e53 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:47.586005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5f006-2971-40d6-a0f1-ae784ee12e53 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:47.587937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 169/326] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:47.592867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d613b7c-1a4d-457e-b080-8b8e7a6da2ee None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:47.593150 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d613b7c-1a4d-457e-b080-8b8e7a6da2ee None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:47.593405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d613b7c-1a4d-457e-b080-8b8e7a6da2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:47.611880 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6d613b7c-1a4d-457e-b080-8b8e7a6da2ee None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:47.613068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6d613b7c-1a4d-457e-b080-8b8e7a6da2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:47.650142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ffe7c-d727-4c54-afde-b6147a71a6c8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:47.679766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 170/327] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:47.680823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d613b7c-1a4d-457e-b080-8b8e7a6da2ee None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:47.683122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:47.690268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220c12b-f83f-4380-b1dc-68f64c82acfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:47.692936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220c12b-f83f-4380-b1dc-68f64c82acfe None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:47.692936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220c12b-f83f-4380-b1dc-68f64c82acfe None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:47.692936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220c12b-f83f-4380-b1dc-68f64c82acfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:47.692936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220c12b-f83f-4380-b1dc-68f64c82acfe None admin] RBAC: Adding query filter params (role.id=08a7c861d84b43d7a4da431f31f3c2d7, user.id=5a6d1e56e8184121b5aba03ff3ef567e, scope.system=all) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:47.692936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220c12b-f83f-4380-b1dc-68f64c82acfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:47.694092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220c12b-f83f-4380-b1dc-68f64c82acfe None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:47.706602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 158/328] 10.222.0.57 () {56 vars in 1308 bytes} [Tue Aug 9 18:29:47 2022] GET /identity/v3/role_assignments?user.id=5a6d1e56e8184121b5aba03ff3ef567e&scope.system=all&role.id=08a7c861d84b43d7a4da431f31f3c2d7 => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:47.707282 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220c12b-f83f-4380-b1dc-68f64c82acfe None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:47.799225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d613b7c-1a4d-457e-b080-8b8e7a6da2ee None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:47.816488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:47.822554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220c12b-f83f-4380-b1dc-68f64c82acfe None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:47.841205 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611ed5d-11d5-4c21-8e6b-84a1b5dcd646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:47.841795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611ed5d-11d5-4c21-8e6b-84a1b5dcd646 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:47.842072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611ed5d-11d5-4c21-8e6b-84a1b5dcd646 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:47.842367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611ed5d-11d5-4c21-8e6b-84a1b5dcd646 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:47.854806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611ed5d-11d5-4c21-8e6b-84a1b5dcd646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-2611ed5d-11d5-4c21-8e6b-84a1b5dcd646 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:47.859682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:47.860548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:47.861308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 171/329] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:47.867409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73212d88-c2d9-4d5d-85c5-9952efed9972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:47.867793 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611ed5d-11d5-4c21-8e6b-84a1b5dcd646 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:47.868251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73212d88-c2d9-4d5d-85c5-9952efed9972 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:47.868561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73212d88-c2d9-4d5d-85c5-9952efed9972 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:47.868828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73212d88-c2d9-4d5d-85c5-9952efed9972 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:47.870214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73212d88-c2d9-4d5d-85c5-9952efed9972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:47.871497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73212d88-c2d9-4d5d-85c5-9952efed9972 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:47.873554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 159/330] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:29:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:47.879218 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73212d88-c2d9-4d5d-85c5-9952efed9972 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:47.941996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:47.981292 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611ed5d-11d5-4c21-8e6b-84a1b5dcd646 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:47.984567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73212d88-c2d9-4d5d-85c5-9952efed9972 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:48.014373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f8d45-52d6-4d90-b469-2635e49b76a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:48.014980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f8d45-52d6-4d90-b469-2635e49b76a1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:48.015258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f8d45-52d6-4d90-b469-2635e49b76a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:48.015531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f8d45-52d6-4d90-b469-2635e49b76a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:48.016187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f8d45-52d6-4d90-b469-2635e49b76a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:48.016489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f8d45-52d6-4d90-b469-2635e49b76a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:48.017577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34490441-dbda-4dd2-84df-53d0e2f90103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:48.017903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f8d45-52d6-4d90-b469-2635e49b76a1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:48.018318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34490441-dbda-4dd2-84df-53d0e2f90103 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:48.018598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34490441-dbda-4dd2-84df-53d0e2f90103 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:48.018867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34490441-dbda-4dd2-84df-53d0e2f90103 None admin] PATH_INFO: `/v3/groups` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:48.019472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34490441-dbda-4dd2-84df-53d0e2f90103 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:48.020927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34490441-dbda-4dd2-84df-53d0e2f90103 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:48.026536 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 172/331] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:48.036355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f8d45-52d6-4d90-b469-2635e49b76a1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:48.129405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f8d45-52d6-4d90-b469-2635e49b76a1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:48.136115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 160/332] 10.222.0.57 () {60 vars in 1134 bytes} [Tue Aug 9 18:29:47 2022] POST /identity/v3/groups => generated 226 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:29:48.158519 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:48.169898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2d85f-24db-44df-a186-f07a6303c347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:48.170556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2d85f-24db-44df-a186-f07a6303c347 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:48.170835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2d85f-24db-44df-a186-f07a6303c347 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:48.171106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2d85f-24db-44df-a186-f07a6303c347 None admin] PATH_INFO: `/v3/users/e0d66d329fec44d2b1d41e8363db9fbf` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:48.172328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2d85f-24db-44df-a186-f07a6303c347 None admin] RBAC: Authorizing `identity:get_user(user_id=e0d66d329fec44d2b1d41e8363db9fbf)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:48.173852 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2d85f-24db-44df-a186-f07a6303c347 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:48.175608 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 173/333] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:48 2022] GET /identity/v3/users/e0d66d329fec44d2b1d41e8363db9fbf => generated 293 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:48.181866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34490441-dbda-4dd2-84df-53d0e2f90103 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:48.210464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:48.303105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34490441-dbda-4dd2-84df-53d0e2f90103 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:48.350231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837b100-62d0-45c9-a026-06026d9ba0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:48.350430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b100-62d0-45c9-a026-06026d9ba0c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:48.350690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b100-62d0-45c9-a026-06026d9ba0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:48.350914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b100-62d0-45c9-a026-06026d9ba0c5 None admin] PATH_INFO: `/v3/system/users/e0d66d329fec44d2b1d41e8363db9fbf/roles/1b625e8349d04de08806337ee4bd9478` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:48.352371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837b100-62d0-45c9-a026-06026d9ba0c5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e0d66d329fec44d2b1d41e8363db9fbf, role_id=1b625e8349d04de08806337ee4bd9478)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:48.356075 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837b100-62d0-45c9-a026-06026d9ba0c5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:48.422211 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 161/334] 10.222.0.57 () {58 vars in 1334 bytes} [Tue Aug 9 18:29:48 2022] PUT /identity/v3/system/users/e0d66d329fec44d2b1d41e8363db9fbf/roles/1b625e8349d04de08806337ee4bd9478 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:29:50.623522 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92b98e-cccb-4695-b09b-a17a35c9c44a None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:50.624059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92b98e-cccb-4695-b09b-a17a35c9c44a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:50.624059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92b98e-cccb-4695-b09b-a17a35c9c44a None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:50.625576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 174/335] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:50.630900 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ae2c8-5bf3-455d-9e99-f4e543d23a03 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:50.631360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ae2c8-5bf3-455d-9e99-f4e543d23a03 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:50.631433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ae2c8-5bf3-455d-9e99-f4e543d23a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:50.651031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-024ae2c8-5bf3-455d-9e99-f4e543d23a03 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:50.652230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-024ae2c8-5bf3-455d-9e99-f4e543d23a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:50.728755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 162/336] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:50.752331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92b98e-cccb-4695-b09b-a17a35c9c44a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:50.867782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92b98e-cccb-4695-b09b-a17a35c9c44a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:50.896798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75fbe5-3596-4cae-8e5b-bbf99772a2a3 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:50.897070 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75fbe5-3596-4cae-8e5b-bbf99772a2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:50.897275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75fbe5-3596-4cae-8e5b-bbf99772a2a3 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:50.898659 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 163/337] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:50.907105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4d561-e6f5-467a-a077-b56e422fe9a2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:50.907396 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4d561-e6f5-467a-a077-b56e422fe9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:50.907680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4d561-e6f5-467a-a077-b56e422fe9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:50.917387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54149d57-3b4e-4573-bd45-980eb127221a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:50.918051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54149d57-3b4e-4573-bd45-980eb127221a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:50.918335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54149d57-3b4e-4573-bd45-980eb127221a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:50.918611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54149d57-3b4e-4573-bd45-980eb127221a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:50.942496 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54149d57-3b4e-4573-bd45-980eb127221a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:50.942683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-54f4d561-e6f5-467a-a077-b56e422fe9a2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:50.943757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-54f4d561-e6f5-467a-a077-b56e422fe9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-54149d57-3b4e-4573-bd45-980eb127221a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:50.948112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:50.948969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:50.949818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 175/338] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:50.956102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54149d57-3b4e-4573-bd45-980eb127221a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:50.972371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:51.046714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:51.086659 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 164/339] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:51.105076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54149d57-3b4e-4573-bd45-980eb127221a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:51.143246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4d561-e6f5-467a-a077-b56e422fe9a2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:51.160993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657d867-9527-4e6b-afa6-fc068b59b780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:51.161899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657d867-9527-4e6b-afa6-fc068b59b780 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:51.162340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657d867-9527-4e6b-afa6-fc068b59b780 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:51.162621 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657d867-9527-4e6b-afa6-fc068b59b780 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:51.163437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657d867-9527-4e6b-afa6-fc068b59b780 None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:51.163744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657d867-9527-4e6b-afa6-fc068b59b780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:51.165477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657d867-9527-4e6b-afa6-fc068b59b780 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:51.177242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 176/340] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:51.183338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657d867-9527-4e6b-afa6-fc068b59b780 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:51.383506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4d561-e6f5-467a-a077-b56e422fe9a2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:51.419954 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657d867-9527-4e6b-afa6-fc068b59b780 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:51.447000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:51.456604 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b93025-9261-4ef4-bd0e-2132c19fb2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:51.462903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b93025-9261-4ef4-bd0e-2132c19fb2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:51.462903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b93025-9261-4ef4-bd0e-2132c19fb2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:51.462903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b93025-9261-4ef4-bd0e-2132c19fb2f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:51.476768 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b93025-9261-4ef4-bd0e-2132c19fb2f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-59b93025-9261-4ef4-bd0e-2132c19fb2f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:51.481646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:51.482499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:51.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:51.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:51.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:51.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:51.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:51.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:51.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:51.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:51.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:51.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:51.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 18:29:51.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:51.483855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 165/341] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:51.488327 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a64f75-5adc-4495-b6d9-175600e3566b None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:51.488609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a64f75-5adc-4495-b6d9-175600e3566b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:51.488885 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a64f75-5adc-4495-b6d9-175600e3566b None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:51.495268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a16f3-792f-4f01-b5a4-7a8dbb19b0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:51.495907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a16f3-792f-4f01-b5a4-7a8dbb19b0fd None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:51.496191 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a16f3-792f-4f01-b5a4-7a8dbb19b0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:51.496462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a16f3-792f-4f01-b5a4-7a8dbb19b0fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:51.499557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 166/342] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:51 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:51.500425 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a64f75-5adc-4495-b6d9-175600e3566b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:51.529773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:51.604704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a16f3-792f-4f01-b5a4-7a8dbb19b0fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:51.606473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a16f3-792f-4f01-b5a4-7a8dbb19b0fd None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:51.655578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a64f75-5adc-4495-b6d9-175600e3566b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:51.694322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d069c-e866-4114-b88d-563613fb5c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:51.694862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d069c-e866-4114-b88d-563613fb5c1b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:51.695144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d069c-e866-4114-b88d-563613fb5c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:51.695418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d069c-e866-4114-b88d-563613fb5c1b None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:51.696249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d069c-e866-4114-b88d-563613fb5c1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:51.697257 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d069c-e866-4114-b88d-563613fb5c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:51.697848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d069c-e866-4114-b88d-563613fb5c1b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:51.706680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 167/343] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:51.707890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9ad9d-366d-45e9-bc02-5da85cca6153 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:51.708242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9ad9d-366d-45e9-bc02-5da85cca6153 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:51.708521 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9ad9d-366d-45e9-bc02-5da85cca6153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:51.722388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 177/344] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:51.724396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618bd11-6394-418b-aa3a-faa3644bfb2e None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:51.724859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618bd11-6394-418b-aa3a-faa3644bfb2e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:51.725301 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618bd11-6394-418b-aa3a-faa3644bfb2e None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:51.726771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 178/345] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:51.728266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618bd11-6394-418b-aa3a-faa3644bfb2e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:51.742888 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d2e9ad9d-366d-45e9-bc02-5da85cca6153 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:51.746483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d2e9ad9d-366d-45e9-bc02-5da85cca6153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:51.749688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:51.829171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:51.840015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 168/346] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:51.845584 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9ad9d-366d-45e9-bc02-5da85cca6153 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:51.908345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618bd11-6394-418b-aa3a-faa3644bfb2e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:51.951397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b9e3d-8491-4d82-90b7-49949e6aef8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:51.952210 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b9e3d-8491-4d82-90b7-49949e6aef8f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:51.952386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b9e3d-8491-4d82-90b7-49949e6aef8f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:51.952634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b9e3d-8491-4d82-90b7-49949e6aef8f None admin] PATH_INFO: `/v3/users/e0d66d329fec44d2b1d41e8363db9fbf` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:51.954152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b9e3d-8491-4d82-90b7-49949e6aef8f None admin] RBAC: Authorizing `identity:get_user(user_id=e0d66d329fec44d2b1d41e8363db9fbf)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:51.955797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b9e3d-8491-4d82-90b7-49949e6aef8f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:51.957721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 179/347] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:29:51 2022] GET /identity/v3/users/e0d66d329fec44d2b1d41e8363db9fbf => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:51.958622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e93a6b-666f-44bc-8862-2c524e306ba3 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:51.958898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e93a6b-666f-44bc-8862-2c524e306ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:51.959169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e93a6b-666f-44bc-8862-2c524e306ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:51.983018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b2e93a6b-666f-44bc-8862-2c524e306ba3 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:51.984186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b2e93a6b-666f-44bc-8862-2c524e306ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:51.988814 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9ad9d-366d-45e9-bc02-5da85cca6153 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:52.034999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f75d27d-a2be-403e-a274-d3df3a303a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:52.035631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75d27d-a2be-403e-a274-d3df3a303a95 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:52.035906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75d27d-a2be-403e-a274-d3df3a303a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:52.036172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75d27d-a2be-403e-a274-d3df3a303a95 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:52.038250 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f75d27d-a2be-403e-a274-d3df3a303a95 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:52.041020 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f75d27d-a2be-403e-a274-d3df3a303a95 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:52.045062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 169/348] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:51 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:52.045062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f75d27d-a2be-403e-a274-d3df3a303a95 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:52.074733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 180/349] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:52.076380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916c8ce-8c14-4544-80c0-bfbf994895e3 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:52.076866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916c8ce-8c14-4544-80c0-bfbf994895e3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:52.077355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916c8ce-8c14-4544-80c0-bfbf994895e3 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:52.079063 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 181/350] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:52.079950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916c8ce-8c14-4544-80c0-bfbf994895e3 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:52.106255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:52.125759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:52.170799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f75d27d-a2be-403e-a274-d3df3a303a95 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:52.219308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916c8ce-8c14-4544-80c0-bfbf994895e3 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:52.219540 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453959d-f7b5-4e5e-a15b-11dc291143cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:52.220182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453959d-f7b5-4e5e-a15b-11dc291143cf None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:52.220465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453959d-f7b5-4e5e-a15b-11dc291143cf None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:52.220740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453959d-f7b5-4e5e-a15b-11dc291143cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:52.235937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453959d-f7b5-4e5e-a15b-11dc291143cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-f453959d-f7b5-4e5e-a15b-11dc291143cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:52.241309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:52.242268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:52.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 170/351] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:52.243800 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453959d-f7b5-4e5e-a15b-11dc291143cf None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:52.276274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb28501-644d-4031-bcbf-ae57834b1fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:52.277137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb28501-644d-4031-bcbf-ae57834b1fdc None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:52.277660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb28501-644d-4031-bcbf-ae57834b1fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:52.277941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb28501-644d-4031-bcbf-ae57834b1fdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:52.278679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb28501-644d-4031-bcbf-ae57834b1fdc None admin] RBAC: Adding query filter params (role.id=1b625e8349d04de08806337ee4bd9478, user.id=e0d66d329fec44d2b1d41e8363db9fbf, scope.system=all) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:52.278991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb28501-644d-4031-bcbf-ae57834b1fdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:52.280451 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb28501-644d-4031-bcbf-ae57834b1fdc None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:52.300920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 182/352] 10.222.0.57 () {56 vars in 1308 bytes} [Tue Aug 9 18:29:52 2022] GET /identity/v3/role_assignments?user.id=e0d66d329fec44d2b1d41e8363db9fbf&scope.system=all&role.id=1b625e8349d04de08806337ee4bd9478 => generated 510 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:52.302346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71285d-9748-4188-bb11-f19fd01f2fd7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:52.302639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71285d-9748-4188-bb11-f19fd01f2fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:52.303815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71285d-9748-4188-bb11-f19fd01f2fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:52.329184 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0c71285d-9748-4188-bb11-f19fd01f2fd7 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:52.330534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0c71285d-9748-4188-bb11-f19fd01f2fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:52.385596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453959d-f7b5-4e5e-a15b-11dc291143cf None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:52.409333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:52.415573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 183/353] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:52.416708 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71285d-9748-4188-bb11-f19fd01f2fd7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:52.424839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:52.437117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b553c3-a25c-4a78-9d23-77e268e81c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:52.437795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b553c3-a25c-4a78-9d23-77e268e81c09 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:52.438078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b553c3-a25c-4a78-9d23-77e268e81c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:52.438350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b553c3-a25c-4a78-9d23-77e268e81c09 None admin] PATH_INFO: `/v3/groups/c336ec08b15c4352a280415e31a2fee8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:52.445105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b553c3-a25c-4a78-9d23-77e268e81c09 None admin] RBAC: Authorizing `identity:get_group(group_id=c336ec08b15c4352a280415e31a2fee8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:52.447274 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b553c3-a25c-4a78-9d23-77e268e81c09 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:52.449329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 171/354] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:29:52 2022] GET /identity/v3/groups/c336ec08b15c4352a280415e31a2fee8 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:52.451589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b553c3-a25c-4a78-9d23-77e268e81c09 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:52.573434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71285d-9748-4188-bb11-f19fd01f2fd7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:52.581832 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b553c3-a25c-4a78-9d23-77e268e81c09 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:52.620171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf99e04-2793-4098-9ba6-3d9affb83629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:52.620815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf99e04-2793-4098-9ba6-3d9affb83629 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:52.621096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf99e04-2793-4098-9ba6-3d9affb83629 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:52.621368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf99e04-2793-4098-9ba6-3d9affb83629 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:52.632182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b200f4-15a3-4abb-b3ab-8988dc7c5aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:52.633136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b200f4-15a3-4abb-b3ab-8988dc7c5aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:52.633464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b200f4-15a3-4abb-b3ab-8988dc7c5aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:52.634331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b200f4-15a3-4abb-b3ab-8988dc7c5aa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:52.634512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf99e04-2793-4098-9ba6-3d9affb83629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:52.635290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b200f4-15a3-4abb-b3ab-8988dc7c5aa2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:52.635597 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b200f4-15a3-4abb-b3ab-8988dc7c5aa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:52.636986 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b200f4-15a3-4abb-b3ab-8988dc7c5aa2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-6cf99e04-2793-4098-9ba6-3d9affb83629 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:52.639763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:52.640640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:52.641428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 184/355] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:29:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:52.643205 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf99e04-2793-4098-9ba6-3d9affb83629 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:52.656293 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 172/356] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:52.657872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b200f4-15a3-4abb-b3ab-8988dc7c5aa2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:52.723564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:52.726450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:52.767952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf99e04-2793-4098-9ba6-3d9affb83629 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:52.781890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b200f4-15a3-4abb-b3ab-8988dc7c5aa2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:52.811292 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf5ec6-5bbc-41c6-a057-fe9c58005fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:52.811910 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf5ec6-5bbc-41c6-a057-fe9c58005fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:52.812196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf5ec6-5bbc-41c6-a057-fe9c58005fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:52.812470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf5ec6-5bbc-41c6-a057-fe9c58005fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:52.813204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf5ec6-5bbc-41c6-a057-fe9c58005fa1 None admin] RBAC: Adding query filter params (group.id=c336ec08b15c4352a280415e31a2fee8, role.id=08a7c861d84b43d7a4da431f31f3c2d7, scope.project.id=5d18db22ee924d578a2f7079502940eb) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:52.813515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf5ec6-5bbc-41c6-a057-fe9c58005fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:52.815074 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf5ec6-5bbc-41c6-a057-fe9c58005fa1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:52.821197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 185/357] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 9 18:29:52 2022] GET /identity/v3/role_assignments?group.id=c336ec08b15c4352a280415e31a2fee8&scope.project.id=5d18db22ee924d578a2f7079502940eb&role.id=08a7c861d84b43d7a4da431f31f3c2d7 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:52.823549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf5ec6-5bbc-41c6-a057-fe9c58005fa1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:52.827852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aaf81a-5c7d-451b-8e78-de934d1aeed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:52.828611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aaf81a-5c7d-451b-8e78-de934d1aeed7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:52.828895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aaf81a-5c7d-451b-8e78-de934d1aeed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:52.829167 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aaf81a-5c7d-451b-8e78-de934d1aeed7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:52.842142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aaf81a-5c7d-451b-8e78-de934d1aeed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-14aaf81a-5c7d-451b-8e78-de934d1aeed7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:52.848064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:52.849098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:52.849950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 173/358] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:29:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:52.850627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aaf81a-5c7d-451b-8e78-de934d1aeed7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:52.957095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf5ec6-5bbc-41c6-a057-fe9c58005fa1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:52.988899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:53.002989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aaf81a-5c7d-451b-8e78-de934d1aeed7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:53.006143 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff999044-ab3c-4a9b-b01e-cb128560cf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:53.007642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff999044-ab3c-4a9b-b01e-cb128560cf9d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:53.007925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff999044-ab3c-4a9b-b01e-cb128560cf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:53.008199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff999044-ab3c-4a9b-b01e-cb128560cf9d None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:53.008993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff999044-ab3c-4a9b-b01e-cb128560cf9d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:53.013061 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff999044-ab3c-4a9b-b01e-cb128560cf9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:53.013061 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff999044-ab3c-4a9b-b01e-cb128560cf9d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:53.020553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 186/359] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:29:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:53.021382 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:53.026864 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ccf66-5497-4f62-85cd-3435ce5298f9 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:53.027121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ccf66-5497-4f62-85cd-3435ce5298f9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:53.027402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ccf66-5497-4f62-85cd-3435ce5298f9 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:53.028715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 187/360] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:53.029482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ccf66-5497-4f62-85cd-3435ce5298f9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:53.050122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c1286-8675-4c51-8914-1bd954d764ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:53.051133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c1286-8675-4c51-8914-1bd954d764ff None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:53.051517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c1286-8675-4c51-8914-1bd954d764ff None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:53.051907 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c1286-8675-4c51-8914-1bd954d764ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:53.145429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c1286-8675-4c51-8914-1bd954d764ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:53.146870 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c1286-8675-4c51-8914-1bd954d764ff None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:53.158435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ccf66-5497-4f62-85cd-3435ce5298f9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:53.202418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f91b9-f80b-4f48-82f4-08fb810b5e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:53.203093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f91b9-f80b-4f48-82f4-08fb810b5e43 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:53.203390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f91b9-f80b-4f48-82f4-08fb810b5e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:53.203664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f91b9-f80b-4f48-82f4-08fb810b5e43 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:53.204449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f91b9-f80b-4f48-82f4-08fb810b5e43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:53.204872 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f91b9-f80b-4f48-82f4-08fb810b5e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:53.206865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f91b9-f80b-4f48-82f4-08fb810b5e43 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:53.216681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 188/361] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:29:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:53.217635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f91b9-f80b-4f48-82f4-08fb810b5e43 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:53.278044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:53.282882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 174/362] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:53.282882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c945f-1224-4529-9860-0e3a09459787 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:53.282882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c945f-1224-4529-9860-0e3a09459787 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:53.282882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c945f-1224-4529-9860-0e3a09459787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:53.307283 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-550c945f-1224-4529-9860-0e3a09459787 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:53.308503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-550c945f-1224-4529-9860-0e3a09459787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:53.324559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:53.349082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f91b9-f80b-4f48-82f4-08fb810b5e43 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:53.389884 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88e325-0ecf-43fa-b9ac-5502f67ce260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:53.390524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88e325-0ecf-43fa-b9ac-5502f67ce260 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:53.390808 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88e325-0ecf-43fa-b9ac-5502f67ce260 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:53.391080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88e325-0ecf-43fa-b9ac-5502f67ce260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:53.398758 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 175/363] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:53.400168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c945f-1224-4529-9860-0e3a09459787 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:53.476858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88e325-0ecf-43fa-b9ac-5502f67ce260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:53.478589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88e325-0ecf-43fa-b9ac-5502f67ce260 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:53.530311 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c945f-1224-4529-9860-0e3a09459787 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:53.545485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:53.593882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f938e9b-d1ec-4a8e-ba05-5fc77363d45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:53.595241 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f938e9b-d1ec-4a8e-ba05-5fc77363d45a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:53.595241 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f938e9b-d1ec-4a8e-ba05-5fc77363d45a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:53.595434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f938e9b-d1ec-4a8e-ba05-5fc77363d45a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:53.620153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 189/364] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:53.625703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88e325-0ecf-43fa-b9ac-5502f67ce260 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:53.629137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:53.705944 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f938e9b-d1ec-4a8e-ba05-5fc77363d45a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:53.707330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f938e9b-d1ec-4a8e-ba05-5fc77363d45a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:53.754077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88e325-0ecf-43fa-b9ac-5502f67ce260 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:53.800277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3d922-4593-490c-9005-6f4d642d6722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:53.800277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d922-4593-490c-9005-6f4d642d6722 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:53.800788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d922-4593-490c-9005-6f4d642d6722 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:53.801068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d922-4593-490c-9005-6f4d642d6722 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:53.802614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3d922-4593-490c-9005-6f4d642d6722 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:53.803742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3d922-4593-490c-9005-6f4d642d6722 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:53.808181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 190/365] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:53 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:53.808181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3d922-4593-490c-9005-6f4d642d6722 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:53.833107 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:53.838048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 176/366] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:53.839182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f938e9b-d1ec-4a8e-ba05-5fc77363d45a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:53.912002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:53.952651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3d922-4593-490c-9005-6f4d642d6722 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:53.981163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f938e9b-d1ec-4a8e-ba05-5fc77363d45a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:54.011321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcd41d-665c-4dd8-bf74-542a667e9c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:54.012163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcd41d-665c-4dd8-bf74-542a667e9c70 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:54.012458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcd41d-665c-4dd8-bf74-542a667e9c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:54.013152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcd41d-665c-4dd8-bf74-542a667e9c70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:54.026223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e426c7f-1f4f-4e78-8c68-1e84cebe7dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:54.026857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426c7f-1f4f-4e78-8c68-1e84cebe7dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:54.027139 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426c7f-1f4f-4e78-8c68-1e84cebe7dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:54.027429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426c7f-1f4f-4e78-8c68-1e84cebe7dd0 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:54.028360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbcd41d-665c-4dd8-bf74-542a667e9c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:54.029078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e426c7f-1f4f-4e78-8c68-1e84cebe7dd0 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:54.032422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e426c7f-1f4f-4e78-8c68-1e84cebe7dd0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:54.032675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 177/367] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:53 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:54.033607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e426c7f-1f4f-4e78-8c68-1e84cebe7dd0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-1bbcd41d-665c-4dd8-bf74-542a667e9c70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:54.033713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:54.034514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:54.035749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 191/368] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:29:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:54.040427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcd41d-665c-4dd8-bf74-542a667e9c70 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:54.139339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:54.172972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcd41d-665c-4dd8-bf74-542a667e9c70 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:54.182115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e426c7f-1f4f-4e78-8c68-1e84cebe7dd0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:54.201213 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:54.216707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511b614-c2e2-4c79-8878-014e2143ffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:54.217345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b614-c2e2-4c79-8878-014e2143ffbb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:54.217661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b614-c2e2-4c79-8878-014e2143ffbb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:54.217947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b614-c2e2-4c79-8878-014e2143ffbb None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:54.223453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511b614-c2e2-4c79-8878-014e2143ffbb None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:54.223453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511b614-c2e2-4c79-8878-014e2143ffbb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:54.223453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 192/369] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:54 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:54.223713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511b614-c2e2-4c79-8878-014e2143ffbb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:54.233549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ff2ef-7b2b-4803-8b9b-08e510167825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:54.234935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ff2ef-7b2b-4803-8b9b-08e510167825 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:54.235402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ff2ef-7b2b-4803-8b9b-08e510167825 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:54.235830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ff2ef-7b2b-4803-8b9b-08e510167825 None admin] PATH_INFO: `/v3/groups/c336ec08b15c4352a280415e31a2fee8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:54.237583 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ff2ef-7b2b-4803-8b9b-08e510167825 None admin] RBAC: Authorizing `identity:get_group(group_id=c336ec08b15c4352a280415e31a2fee8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:54.240574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ff2ef-7b2b-4803-8b9b-08e510167825 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:54.242595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 178/370] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:29:54 2022] GET /identity/v3/groups/c336ec08b15c4352a280415e31a2fee8 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:54.244908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ff2ef-7b2b-4803-8b9b-08e510167825 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:54.360496 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511b614-c2e2-4c79-8878-014e2143ffbb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:54.384729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ff2ef-7b2b-4803-8b9b-08e510167825 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:54.401114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6155922-2e5d-44f5-ad34-aaa23fff0c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:54.401928 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6155922-2e5d-44f5-ad34-aaa23fff0c1d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:54.402327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6155922-2e5d-44f5-ad34-aaa23fff0c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:54.402620 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6155922-2e5d-44f5-ad34-aaa23fff0c1d None admin] PATH_INFO: `/v3/groups/c336ec08b15c4352a280415e31a2fee8` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:54.404424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6155922-2e5d-44f5-ad34-aaa23fff0c1d None admin] RBAC: Authorizing `identity:get_group(group_id=c336ec08b15c4352a280415e31a2fee8)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:54.406204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6155922-2e5d-44f5-ad34-aaa23fff0c1d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:54.408246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 193/371] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:29:54 2022] GET /identity/v3/groups/c336ec08b15c4352a280415e31a2fee8 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:54.409080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6155922-2e5d-44f5-ad34-aaa23fff0c1d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:54.427110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48232323-7177-4ed8-9e51-46958657f262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:54.427691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:54.429067 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48232323-7177-4ed8-9e51-46958657f262 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:54.429543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48232323-7177-4ed8-9e51-46958657f262 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:54.429795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48232323-7177-4ed8-9e51-46958657f262 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:54.430494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48232323-7177-4ed8-9e51-46958657f262 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:54.430803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48232323-7177-4ed8-9e51-46958657f262 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:54.432374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48232323-7177-4ed8-9e51-46958657f262 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:54.441917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 179/372] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:54.442855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48232323-7177-4ed8-9e51-46958657f262 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:54.491269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:54.526235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6155922-2e5d-44f5-ad34-aaa23fff0c1d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:54.553927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48232323-7177-4ed8-9e51-46958657f262 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:54.566488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f99b58-b3c0-48ff-8f4f-f0e31a93726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:54.567132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f99b58-b3c0-48ff-8f4f-f0e31a93726e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:54.567416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f99b58-b3c0-48ff-8f4f-f0e31a93726e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:54.567689 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f99b58-b3c0-48ff-8f4f-f0e31a93726e None admin] PATH_INFO: `/v3/groups/c336ec08b15c4352a280415e31a2fee8` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:54.568819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f99b58-b3c0-48ff-8f4f-f0e31a93726e None admin] RBAC: Authorizing `identity:get_group(group_id=c336ec08b15c4352a280415e31a2fee8)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:54.570309 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f99b58-b3c0-48ff-8f4f-f0e31a93726e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:54.572050 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 194/373] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:29:54 2022] GET /identity/v3/groups/c336ec08b15c4352a280415e31a2fee8 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:54.577311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f99b58-b3c0-48ff-8f4f-f0e31a93726e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:54.589517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652775ca-ee0f-4a74-9695-32e79830b7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:54.590143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652775ca-ee0f-4a74-9695-32e79830b7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:54.590428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652775ca-ee0f-4a74-9695-32e79830b7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:54.590706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652775ca-ee0f-4a74-9695-32e79830b7b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:54.591434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652775ca-ee0f-4a74-9695-32e79830b7b6 None admin] RBAC: Adding query filter params (group.id=c336ec08b15c4352a280415e31a2fee8, role.id=08a7c861d84b43d7a4da431f31f3c2d7, scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:54.591741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652775ca-ee0f-4a74-9695-32e79830b7b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:54.593171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652775ca-ee0f-4a74-9695-32e79830b7b6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:54.609554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 180/374] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 9 18:29:54 2022] GET /identity/v3/role_assignments?group.id=c336ec08b15c4352a280415e31a2fee8&scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85&role.id=08a7c861d84b43d7a4da431f31f3c2d7 => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:54.609554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652775ca-ee0f-4a74-9695-32e79830b7b6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:54.699127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:54.720925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f99b58-b3c0-48ff-8f4f-f0e31a93726e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:54.749485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652775ca-ee0f-4a74-9695-32e79830b7b6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:54.757814 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:54.768799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be9c48-a796-4c92-acf1-3db34ca1d3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:54.769989 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be9c48-a796-4c92-acf1-3db34ca1d3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:54.770269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be9c48-a796-4c92-acf1-3db34ca1d3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:54.770536 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be9c48-a796-4c92-acf1-3db34ca1d3a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:54.775021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be9c48-a796-4c92-acf1-3db34ca1d3a1 None admin] RBAC: Adding query filter params (group.id=c336ec08b15c4352a280415e31a2fee8, role.id=1d1b5b96118040f9a4fd019bbcd93071, scope.project.id=5d18db22ee924d578a2f7079502940eb) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:54.775021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be9c48-a796-4c92-acf1-3db34ca1d3a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:54.775021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be9c48-a796-4c92-acf1-3db34ca1d3a1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:54.783492 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 195/375] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 9 18:29:54 2022] GET /identity/v3/role_assignments?group.id=c336ec08b15c4352a280415e31a2fee8&scope.project.id=5d18db22ee924d578a2f7079502940eb&role.id=1d1b5b96118040f9a4fd019bbcd93071 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:54.784537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be9c48-a796-4c92-acf1-3db34ca1d3a1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:54.805217 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf6933-68c6-435d-8d8a-ae2a19e2a526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:54.806246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf6933-68c6-435d-8d8a-ae2a19e2a526 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:54.806739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf6933-68c6-435d-8d8a-ae2a19e2a526 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:54.808092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf6933-68c6-435d-8d8a-ae2a19e2a526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:54.961057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf6933-68c6-435d-8d8a-ae2a19e2a526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:54.963152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf6933-68c6-435d-8d8a-ae2a19e2a526 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:54.987363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be9c48-a796-4c92-acf1-3db34ca1d3a1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:55.023913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a270994-9ada-4a80-bd1d-d3d953329788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:55.025000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a270994-9ada-4a80-bd1d-d3d953329788 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:55.025415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a270994-9ada-4a80-bd1d-d3d953329788 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:55.025738 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a270994-9ada-4a80-bd1d-d3d953329788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:55.026911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a270994-9ada-4a80-bd1d-d3d953329788 None admin] RBAC: Adding query filter params (group.id=c336ec08b15c4352a280415e31a2fee8, role.id=1d1b5b96118040f9a4fd019bbcd93071, scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:55.027225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a270994-9ada-4a80-bd1d-d3d953329788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:55.028966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a270994-9ada-4a80-bd1d-d3d953329788 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:55.038079 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 196/376] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 9 18:29:54 2022] GET /identity/v3/role_assignments?group.id=c336ec08b15c4352a280415e31a2fee8&scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85&role.id=1d1b5b96118040f9a4fd019bbcd93071 => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:55.040844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138dd5b-fa03-46a1-9b0e-08886e942f47 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:55.041184 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138dd5b-fa03-46a1-9b0e-08886e942f47 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:55.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138dd5b-fa03-46a1-9b0e-08886e942f47 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:55.043623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 197/377] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:55.050475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3609e1-2062-4473-a0bd-c96a74507f88 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:55.050756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3609e1-2062-4473-a0bd-c96a74507f88 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:55.051031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3609e1-2062-4473-a0bd-c96a74507f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:55.079298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8e3609e1-2062-4473-a0bd-c96a74507f88 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:55.079298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8e3609e1-2062-4473-a0bd-c96a74507f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:55.080674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:55.139126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 181/378] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:29:55.149695 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf6933-68c6-435d-8d8a-ae2a19e2a526 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:55.174820 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:55.229649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 198/379] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:55.284224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3609e1-2062-4473-a0bd-c96a74507f88 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:55.315292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf6933-68c6-435d-8d8a-ae2a19e2a526 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:55.362731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31687a-e004-4f87-9564-4ae7316c7955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:55.363382 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31687a-e004-4f87-9564-4ae7316c7955 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:55.363666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31687a-e004-4f87-9564-4ae7316c7955 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:55.363954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31687a-e004-4f87-9564-4ae7316c7955 None admin] PATH_INFO: `/v3/projects/ee70c279239f4e719fad35784db1a619` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:55.365138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31687a-e004-4f87-9564-4ae7316c7955 None admin] RBAC: Authorizing `identity:get_project(project_id=ee70c279239f4e719fad35784db1a619)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:55.366805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31687a-e004-4f87-9564-4ae7316c7955 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:55.369059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 182/380] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:55 2022] GET /identity/v3/projects/ee70c279239f4e719fad35784db1a619 => generated 344 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:55.375850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31687a-e004-4f87-9564-4ae7316c7955 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:55.453920 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:55.479793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31687a-e004-4f87-9564-4ae7316c7955 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:55.509584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3609e1-2062-4473-a0bd-c96a74507f88 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:55.526650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabe7f4-d584-40a5-ae97-8ec8f6c40c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:55.527305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabe7f4-d584-40a5-ae97-8ec8f6c40c73 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:55.527586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabe7f4-d584-40a5-ae97-8ec8f6c40c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:55.527860 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabe7f4-d584-40a5-ae97-8ec8f6c40c73 None admin] PATH_INFO: `/v3/groups/3e97f57f232f42a3b1a7d172dae1513b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:55.536415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabe7f4-d584-40a5-ae97-8ec8f6c40c73 None admin] RBAC: Authorizing `identity:get_group(group_id=3e97f57f232f42a3b1a7d172dae1513b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:55.538830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabe7f4-d584-40a5-ae97-8ec8f6c40c73 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:55.541062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 183/381] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:29:55 2022] GET /identity/v3/groups/3e97f57f232f42a3b1a7d172dae1513b => generated 229 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:55.549855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabe7f4-d584-40a5-ae97-8ec8f6c40c73 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:55.590785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c6149-3d31-4ea4-8a64-0aa0ebbf7344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:55.591888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c6149-3d31-4ea4-8a64-0aa0ebbf7344 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:55.592404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c6149-3d31-4ea4-8a64-0aa0ebbf7344 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:55.592906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c6149-3d31-4ea4-8a64-0aa0ebbf7344 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:55.616859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:55.624781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c6149-3d31-4ea4-8a64-0aa0ebbf7344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-3d1c6149-3d31-4ea4-8a64-0aa0ebbf7344 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:55.631416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:55.633347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:55.634214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 199/382] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:55.638580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c6149-3d31-4ea4-8a64-0aa0ebbf7344 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:55.709276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabe7f4-d584-40a5-ae97-8ec8f6c40c73 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:55.746530 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8abff-f0e8-4d4a-abe2-bb0ee3a542ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:55.747142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8abff-f0e8-4d4a-abe2-bb0ee3a542ba None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:55.747425 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8abff-f0e8-4d4a-abe2-bb0ee3a542ba None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:55.747697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8abff-f0e8-4d4a-abe2-bb0ee3a542ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:55.748424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8abff-f0e8-4d4a-abe2-bb0ee3a542ba None admin] RBAC: Adding query filter params (group.id=3e97f57f232f42a3b1a7d172dae1513b, role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=ee70c279239f4e719fad35784db1a619) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:55.748794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8abff-f0e8-4d4a-abe2-bb0ee3a542ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:55.750257 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8abff-f0e8-4d4a-abe2-bb0ee3a542ba None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:55.756023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 184/383] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 9 18:29:55 2022] GET /identity/v3/role_assignments?group.id=3e97f57f232f42a3b1a7d172dae1513b&scope.project.id=ee70c279239f4e719fad35784db1a619&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:55.855826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c6149-3d31-4ea4-8a64-0aa0ebbf7344 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:55.910806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334ce00-eab2-46f0-bb56-6540a6f72bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:55.911410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ce00-eab2-46f0-bb56-6540a6f72bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:55.911694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ce00-eab2-46f0-bb56-6540a6f72bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:55.911970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ce00-eab2-46f0-bb56-6540a6f72bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:55.912653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334ce00-eab2-46f0-bb56-6540a6f72bb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:55.912957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334ce00-eab2-46f0-bb56-6540a6f72bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:55.914431 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334ce00-eab2-46f0-bb56-6540a6f72bb8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:55.935998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 200/384] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:55.942460 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8abff-f0e8-4d4a-abe2-bb0ee3a542ba None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:55.945638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:56.069283 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8abff-f0e8-4d4a-abe2-bb0ee3a542ba None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:56.112762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc960c7-e3f6-409c-9b3f-1389201892f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:56.113319 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc960c7-e3f6-409c-9b3f-1389201892f7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:56.113629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc960c7-e3f6-409c-9b3f-1389201892f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:56.113916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc960c7-e3f6-409c-9b3f-1389201892f7 None admin] PATH_INFO: `/v3/groups/c336ec08b15c4352a280415e31a2fee8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:56.115037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc960c7-e3f6-409c-9b3f-1389201892f7 None admin] RBAC: Authorizing `identity:get_group(group_id=c336ec08b15c4352a280415e31a2fee8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:56.116471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc960c7-e3f6-409c-9b3f-1389201892f7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:56.118457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 185/385] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:29:55 2022] GET /identity/v3/groups/c336ec08b15c4352a280415e31a2fee8 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:56.123648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334ce00-eab2-46f0-bb56-6540a6f72bb8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:56.219390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:56.262440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334ce00-eab2-46f0-bb56-6540a6f72bb8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:56.312996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a6bb1-3e1d-41e6-9695-99a7814be492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:56.313663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a6bb1-3e1d-41e6-9695-99a7814be492 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:56.314060 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a6bb1-3e1d-41e6-9695-99a7814be492 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:56.314338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a6bb1-3e1d-41e6-9695-99a7814be492 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:56.318246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a6bb1-3e1d-41e6-9695-99a7814be492 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:56.319708 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a6bb1-3e1d-41e6-9695-99a7814be492 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:56.322218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 201/386] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:56 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:56.327704 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc960c7-e3f6-409c-9b3f-1389201892f7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:56.433867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:56.473885 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc960c7-e3f6-409c-9b3f-1389201892f7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:56.518755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047aaf1-c713-4f55-9bf0-8f22e45283b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:56.519433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047aaf1-c713-4f55-9bf0-8f22e45283b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:56.519697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047aaf1-c713-4f55-9bf0-8f22e45283b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:56.519973 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047aaf1-c713-4f55-9bf0-8f22e45283b2 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb/groups/c336ec08b15c4352a280415e31a2fee8/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:56.523624 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047aaf1-c713-4f55-9bf0-8f22e45283b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d18db22ee924d578a2f7079502940eb, group_id=c336ec08b15c4352a280415e31a2fee8, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:56.525658 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047aaf1-c713-4f55-9bf0-8f22e45283b2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:56.763827 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 186/387] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Aug 9 18:29:56 2022] PUT /identity/v3/projects/5d18db22ee924d578a2f7079502940eb/groups/c336ec08b15c4352a280415e31a2fee8/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:29:57.014599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3406a6-edfb-404b-a8b5-3144e3cb6ab5 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.015983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3406a6-edfb-404b-a8b5-3144e3cb6ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.016478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3406a6-edfb-404b-a8b5-3144e3cb6ab5 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:57.022528 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 202/388] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:57.029692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1891b-1401-47b8-9146-822394186a69 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.033744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbefe6-93da-4842-8d00-4dbd0fdaf51a None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.033874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1891b-1401-47b8-9146-822394186a69 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.033874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1891b-1401-47b8-9146-822394186a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:57.034014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbefe6-93da-4842-8d00-4dbd0fdaf51a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.034014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbefe6-93da-4842-8d00-4dbd0fdaf51a None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:57.034014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 203/389] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:57.039242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56c60b-2dbe-4519-88c1-ab58a0b50c07 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.040252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56c60b-2dbe-4519-88c1-ab58a0b50c07 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.040791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56c60b-2dbe-4519-88c1-ab58a0b50c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:57.052684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-30b1891b-1401-47b8-9146-822394186a69 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:57.053912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-30b1891b-1401-47b8-9146-822394186a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:57.066149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1a56c60b-2dbe-4519-88c1-ab58a0b50c07 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:57.068507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1a56c60b-2dbe-4519-88c1-ab58a0b50c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:57.151715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 187/390] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:57.171951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 204/391] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:29:57.185616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1891b-1401-47b8-9146-822394186a69 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:57.240499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56c60b-2dbe-4519-88c1-ab58a0b50c07 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:57.271799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:57.317160 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1891b-1401-47b8-9146-822394186a69 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:57.332929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:57.356137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b07a6f-62ad-4d43-a3a3-541bc415ed2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:57.357410 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b07a6f-62ad-4d43-a3a3-541bc415ed2a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.357921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b07a6f-62ad-4d43-a3a3-541bc415ed2a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.358348 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b07a6f-62ad-4d43-a3a3-541bc415ed2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:57.371772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b07a6f-62ad-4d43-a3a3-541bc415ed2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-37b07a6f-62ad-4d43-a3a3-541bc415ed2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:57.377151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:57.378315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:57.379316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:57.379316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:57.379316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:57.379316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:57.379316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:57.379316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:57.379316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:57.379316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:57.379316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:57.379316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:57.379316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:57.379316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:57.383087 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 188/392] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:57.388401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56c60b-2dbe-4519-88c1-ab58a0b50c07 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:57.389239 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b07a6f-62ad-4d43-a3a3-541bc415ed2a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:57.424618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c9b4d-ec64-42f2-a2e0-affd93f2a97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:57.425440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c9b4d-ec64-42f2-a2e0-affd93f2a97b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.426148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c9b4d-ec64-42f2-a2e0-affd93f2a97b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.426533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c9b4d-ec64-42f2-a2e0-affd93f2a97b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:57.440317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c9b4d-ec64-42f2-a2e0-affd93f2a97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-e54c9b4d-ec64-42f2-a2e0-affd93f2a97b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:57.447207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:57.448241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:57.449035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:57.449035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:57.449035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:57.449035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:57.449035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:57.449035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:57.449035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:57.449035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:57.449035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:57.449035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:57.449035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:57.449035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:57.461586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 205/393] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:29:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:57.461586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c9b4d-ec64-42f2-a2e0-affd93f2a97b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:57.536723 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b07a6f-62ad-4d43-a3a3-541bc415ed2a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:57.559095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:57.574531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72860a-cc7c-4cfb-8223-730ab827e948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:57.593390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72860a-cc7c-4cfb-8223-730ab827e948 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.593390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72860a-cc7c-4cfb-8223-730ab827e948 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.593390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72860a-cc7c-4cfb-8223-730ab827e948 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:57.593390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72860a-cc7c-4cfb-8223-730ab827e948 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:57.593390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72860a-cc7c-4cfb-8223-730ab827e948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:57.594011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c9b4d-ec64-42f2-a2e0-affd93f2a97b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:57.594093 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72860a-cc7c-4cfb-8223-730ab827e948 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:57.603538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 189/394] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:57.605629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a37b2-41be-4696-8c28-9ff0063a1297 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.606165 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a37b2-41be-4696-8c28-9ff0063a1297 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.606653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a37b2-41be-4696-8c28-9ff0063a1297 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:57.607130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:57.609052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 190/395] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:57.613287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a37b2-41be-4696-8c28-9ff0063a1297 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:57.629768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01b801-725a-45b5-87ba-c29268208ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:57.630388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01b801-725a-45b5-87ba-c29268208ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.630927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01b801-725a-45b5-87ba-c29268208ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.631206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01b801-725a-45b5-87ba-c29268208ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:57.631955 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01b801-725a-45b5-87ba-c29268208ee6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:57.632255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01b801-725a-45b5-87ba-c29268208ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:57.633852 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01b801-725a-45b5-87ba-c29268208ee6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:57.644615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 206/396] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:29:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:57.646724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b350dcf-874b-4d7b-bb12-d774f7a75e64 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.647233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b350dcf-874b-4d7b-bb12-d774f7a75e64 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.647710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b350dcf-874b-4d7b-bb12-d774f7a75e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:57.672514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5b350dcf-874b-4d7b-bb12-d774f7a75e64 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:57.674108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5b350dcf-874b-4d7b-bb12-d774f7a75e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:57.751608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a37b2-41be-4696-8c28-9ff0063a1297 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:57.756106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 207/397] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:57.757171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b350dcf-874b-4d7b-bb12-d774f7a75e64 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:57.797173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3be29f-b4ff-4b2d-b329-4c7eb36b2b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:57.797846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3be29f-b4ff-4b2d-b329-4c7eb36b2b25 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.798032 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3be29f-b4ff-4b2d-b329-4c7eb36b2b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.798304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3be29f-b4ff-4b2d-b329-4c7eb36b2b25 None admin] PATH_INFO: `/v3/groups/c336ec08b15c4352a280415e31a2fee8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:57.799449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3be29f-b4ff-4b2d-b329-4c7eb36b2b25 None admin] RBAC: Authorizing `identity:get_group(group_id=c336ec08b15c4352a280415e31a2fee8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:57.800879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3be29f-b4ff-4b2d-b329-4c7eb36b2b25 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:57.805575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 191/398] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:29:57 2022] GET /identity/v3/groups/c336ec08b15c4352a280415e31a2fee8 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:57.805575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3be29f-b4ff-4b2d-b329-4c7eb36b2b25 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:57.870845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:57.877680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b350dcf-874b-4d7b-bb12-d774f7a75e64 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:57.891057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:57.919120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c79fe-dc43-4dae-a128-4ea5865cbda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:57.920118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c79fe-dc43-4dae-a128-4ea5865cbda8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.920615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c79fe-dc43-4dae-a128-4ea5865cbda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.921199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c79fe-dc43-4dae-a128-4ea5865cbda8 None admin] PATH_INFO: `/v3/groups/c336ec08b15c4352a280415e31a2fee8` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:57.922775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c79fe-dc43-4dae-a128-4ea5865cbda8 None admin] RBAC: Authorizing `identity:get_group(group_id=c336ec08b15c4352a280415e31a2fee8)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:57.924501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c79fe-dc43-4dae-a128-4ea5865cbda8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:57.926779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 208/399] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:29:57 2022] GET /identity/v3/groups/c336ec08b15c4352a280415e31a2fee8 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:57.927817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c79fe-dc43-4dae-a128-4ea5865cbda8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:57.939434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3be29f-b4ff-4b2d-b329-4c7eb36b2b25 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:57.985672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a43ba9-0029-4ff7-9616-db8bbbdb0b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:57.986310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a43ba9-0029-4ff7-9616-db8bbbdb0b50 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:57.986594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a43ba9-0029-4ff7-9616-db8bbbdb0b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:57.986878 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a43ba9-0029-4ff7-9616-db8bbbdb0b50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:58.000803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a43ba9-0029-4ff7-9616-db8bbbdb0b50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-64a43ba9-0029-4ff7-9616-db8bbbdb0b50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:58.007356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:58.008265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:58.009174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 192/400] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:29:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:58.009858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a43ba9-0029-4ff7-9616-db8bbbdb0b50 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:58.062411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c79fe-dc43-4dae-a128-4ea5865cbda8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:58.100637 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abe997-5f81-44de-995e-9bdca609dc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:58.101826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abe997-5f81-44de-995e-9bdca609dc73 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:58.102898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abe997-5f81-44de-995e-9bdca609dc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:58.103369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abe997-5f81-44de-995e-9bdca609dc73 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:58.105643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abe997-5f81-44de-995e-9bdca609dc73 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:58.107935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abe997-5f81-44de-995e-9bdca609dc73 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:58.114843 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 209/401] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:57 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:58.115942 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abe997-5f81-44de-995e-9bdca609dc73 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:58.136038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a43ba9-0029-4ff7-9616-db8bbbdb0b50 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:58.153359 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:58.175781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:58.187626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50a3fb-a8ec-4832-bf57-e7524fd25ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:58.188812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50a3fb-a8ec-4832-bf57-e7524fd25ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:58.189171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50a3fb-a8ec-4832-bf57-e7524fd25ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:58.189571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50a3fb-a8ec-4832-bf57-e7524fd25ec8 None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:58.190777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50a3fb-a8ec-4832-bf57-e7524fd25ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:58.192918 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50a3fb-a8ec-4832-bf57-e7524fd25ec8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:58.197594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 193/402] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:58 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:58.198696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50a3fb-a8ec-4832-bf57-e7524fd25ec8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:58.254091 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abe997-5f81-44de-995e-9bdca609dc73 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:58.297066 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026e0ea-c924-47fc-a97c-f6fd99750475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:58.297728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026e0ea-c924-47fc-a97c-f6fd99750475 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:58.298013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026e0ea-c924-47fc-a97c-f6fd99750475 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:58.298286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026e0ea-c924-47fc-a97c-f6fd99750475 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:58.298953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026e0ea-c924-47fc-a97c-f6fd99750475 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:58.299257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026e0ea-c924-47fc-a97c-f6fd99750475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:58.300644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026e0ea-c924-47fc-a97c-f6fd99750475 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:58.312056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 210/403] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:29:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:58.313470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026e0ea-c924-47fc-a97c-f6fd99750475 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:58.332638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50a3fb-a8ec-4832-bf57-e7524fd25ec8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:58.370841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1011d-c141-4865-b0ec-e07e53033b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:58.371488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1011d-c141-4865-b0ec-e07e53033b21 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:58.371759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1011d-c141-4865-b0ec-e07e53033b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:58.372029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1011d-c141-4865-b0ec-e07e53033b21 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85/groups/c336ec08b15c4352a280415e31a2fee8/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:58.373946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1011d-c141-4865-b0ec-e07e53033b21 None admin] RBAC: Authorizing `identity:create_grant(project_id=38ed82c9c6da41e596c28a3fb28c8c85, group_id=c336ec08b15c4352a280415e31a2fee8, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:58.375373 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1011d-c141-4865-b0ec-e07e53033b21 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:58.425305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026e0ea-c924-47fc-a97c-f6fd99750475 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:58.434218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:58.448288 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 194/404] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Aug 9 18:29:58 2022] PUT /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85/groups/c336ec08b15c4352a280415e31a2fee8/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:29:58.449103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1011d-c141-4865-b0ec-e07e53033b21 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:58.473569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1e183-aa9f-41ed-a1fa-17332ea0808c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:58.474951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1e183-aa9f-41ed-a1fa-17332ea0808c None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:58.475422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1e183-aa9f-41ed-a1fa-17332ea0808c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:58.475983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1e183-aa9f-41ed-a1fa-17332ea0808c None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb/groups/c336ec08b15c4352a280415e31a2fee8/roles/1d1b5b96118040f9a4fd019bbcd93071` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:58.478720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1e183-aa9f-41ed-a1fa-17332ea0808c None admin] RBAC: Authorizing `identity:create_grant(project_id=5d18db22ee924d578a2f7079502940eb, group_id=c336ec08b15c4352a280415e31a2fee8, role_id=1d1b5b96118040f9a4fd019bbcd93071)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:58.480536 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1e183-aa9f-41ed-a1fa-17332ea0808c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:58.505267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:58.553020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 211/405] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Aug 9 18:29:58 2022] PUT /identity/v3/projects/5d18db22ee924d578a2f7079502940eb/groups/c336ec08b15c4352a280415e31a2fee8/roles/1d1b5b96118040f9a4fd019bbcd93071 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:29:58.554502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007e1f3-792e-4414-bb2c-2ee36f19fd5b None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:58.554903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007e1f3-792e-4414-bb2c-2ee36f19fd5b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:58.555296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007e1f3-792e-4414-bb2c-2ee36f19fd5b None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:58.562316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 212/406] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:58.564643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c65159-7285-49da-a051-9be8812e08a8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:58.564934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c65159-7285-49da-a051-9be8812e08a8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:58.565362 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c65159-7285-49da-a051-9be8812e08a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:58.587363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b8c65159-7285-49da-a051-9be8812e08a8 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:58.589002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b8c65159-7285-49da-a051-9be8812e08a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:58.589956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1011d-c141-4865-b0ec-e07e53033b21 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:58.646636 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c7e0b-d738-49f7-9980-c24c941bd1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:58.647465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c7e0b-d738-49f7-9980-c24c941bd1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:58.647838 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c7e0b-d738-49f7-9980-c24c941bd1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:58.648439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c7e0b-d738-49f7-9980-c24c941bd1e9 None admin] PATH_INFO: `/v3/groups/c336ec08b15c4352a280415e31a2fee8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:58.651785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c7e0b-d738-49f7-9980-c24c941bd1e9 None admin] RBAC: Authorizing `identity:get_group(group_id=c336ec08b15c4352a280415e31a2fee8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:58.653756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c7e0b-d738-49f7-9980-c24c941bd1e9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:58.656136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 195/407] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:29:58 2022] GET /identity/v3/groups/c336ec08b15c4352a280415e31a2fee8 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:58.662054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c7e0b-d738-49f7-9980-c24c941bd1e9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:58.691329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 213/408] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:58.715889 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c65159-7285-49da-a051-9be8812e08a8 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:58.783330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c7e0b-d738-49f7-9980-c24c941bd1e9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:58.791184 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:58.838506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86be709-7ffc-4a7d-9577-122367ee8a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:58.839132 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86be709-7ffc-4a7d-9577-122367ee8a53 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:58.839324 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86be709-7ffc-4a7d-9577-122367ee8a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:58.839594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86be709-7ffc-4a7d-9577-122367ee8a53 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:58.843729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86be709-7ffc-4a7d-9577-122367ee8a53 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:58.845176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86be709-7ffc-4a7d-9577-122367ee8a53 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:58.846875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 196/409] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:58 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:58.866803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:58.871326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c65159-7285-49da-a051-9be8812e08a8 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:58.878931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86be709-7ffc-4a7d-9577-122367ee8a53 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:58.956192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67e880-9fce-4a3b-b3d7-35fc15798677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:58.957230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67e880-9fce-4a3b-b3d7-35fc15798677 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:58.957508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67e880-9fce-4a3b-b3d7-35fc15798677 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:58.957794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67e880-9fce-4a3b-b3d7-35fc15798677 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:58.986991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67e880-9fce-4a3b-b3d7-35fc15798677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-ed67e880-9fce-4a3b-b3d7-35fc15798677 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:58.991788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:58.992655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:58.993503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:58.993503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:58.993503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:58.993503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:58.993503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:58.993503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:58.993503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:58.993503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:58.993503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:58.993503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:58.993503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:29:58.993503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:29:59.000377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 214/410] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:29:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:29:59.006158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67e880-9fce-4a3b-b3d7-35fc15798677 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:59.043723 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86be709-7ffc-4a7d-9577-122367ee8a53 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:59.109757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7aa3cf-160b-457c-9d13-684a03a6dd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:59.110874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aa3cf-160b-457c-9d13-684a03a6dd7f None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:59.111493 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aa3cf-160b-457c-9d13-684a03a6dd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:59.111869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aa3cf-160b-457c-9d13-684a03a6dd7f None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85/groups/c336ec08b15c4352a280415e31a2fee8/roles/1d1b5b96118040f9a4fd019bbcd93071` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:59.114243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7aa3cf-160b-457c-9d13-684a03a6dd7f None admin] RBAC: Authorizing `identity:create_grant(project_id=38ed82c9c6da41e596c28a3fb28c8c85, group_id=c336ec08b15c4352a280415e31a2fee8, role_id=1d1b5b96118040f9a4fd019bbcd93071)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:59.115901 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7aa3cf-160b-457c-9d13-684a03a6dd7f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:59.152834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67e880-9fce-4a3b-b3d7-35fc15798677 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:59.184988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 197/411] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Aug 9 18:29:58 2022] PUT /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85/groups/c336ec08b15c4352a280415e31a2fee8/roles/1d1b5b96118040f9a4fd019bbcd93071 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:29:59.187029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c32597-e48b-4898-882f-b6cf8fc9b9a9 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:59.187624 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c32597-e48b-4898-882f-b6cf8fc9b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:59.188287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c32597-e48b-4898-882f-b6cf8fc9b9a9 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:59.191140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 198/412] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:29:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:29:59.197715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b8f07-4598-41e0-aa2e-6af96432cdd7 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:59.198337 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b8f07-4598-41e0-aa2e-6af96432cdd7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:59.198880 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b8f07-4598-41e0-aa2e-6af96432cdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:59.206566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a23f76-5cf9-443b-8697-5368e99641ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:59.207952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a23f76-5cf9-443b-8697-5368e99641ee None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:59.209025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a23f76-5cf9-443b-8697-5368e99641ee None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:59.209827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a23f76-5cf9-443b-8697-5368e99641ee None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:59.210838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a23f76-5cf9-443b-8697-5368e99641ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:59.212602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a23f76-5cf9-443b-8697-5368e99641ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:59.217557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a23f76-5cf9-443b-8697-5368e99641ee None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:59.238200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 215/413] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:59.243824 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a23f76-5cf9-443b-8697-5368e99641ee None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:59.247153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-942b8f07-4598-41e0-aa2e-6af96432cdd7 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:29:59.251536 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-942b8f07-4598-41e0-aa2e-6af96432cdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:29:59.312148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:59.342613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:59.390022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 199/414] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:29:59.417195 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a23f76-5cf9-443b-8697-5368e99641ee None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:59.419891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b8f07-4598-41e0-aa2e-6af96432cdd7 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:59.463470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680bdec-8a2e-49b5-9022-d8ee2a93309e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:59.464490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680bdec-8a2e-49b5-9022-d8ee2a93309e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:59.464976 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680bdec-8a2e-49b5-9022-d8ee2a93309e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:59.465466 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680bdec-8a2e-49b5-9022-d8ee2a93309e None admin] PATH_INFO: `/v3/groups/3e97f57f232f42a3b1a7d172dae1513b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:59.467153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680bdec-8a2e-49b5-9022-d8ee2a93309e None admin] RBAC: Authorizing `identity:get_group(group_id=3e97f57f232f42a3b1a7d172dae1513b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:59.469475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680bdec-8a2e-49b5-9022-d8ee2a93309e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:59.472273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 216/415] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:29:59 2022] GET /identity/v3/groups/3e97f57f232f42a3b1a7d172dae1513b => generated 229 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:29:59.477637 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680bdec-8a2e-49b5-9022-d8ee2a93309e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:59.572163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b8f07-4598-41e0-aa2e-6af96432cdd7 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:59.625503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:59.638982 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680bdec-8a2e-49b5-9022-d8ee2a93309e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:59.657192 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f09437-1af5-4f37-8a76-f964200eb545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:59.657972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f09437-1af5-4f37-8a76-f964200eb545 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:59.658347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f09437-1af5-4f37-8a76-f964200eb545 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:59.658720 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f09437-1af5-4f37-8a76-f964200eb545 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:59.679518 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f09437-1af5-4f37-8a76-f964200eb545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-c9f09437-1af5-4f37-8a76-f964200eb545 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:29:59.685454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:29:59.686413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:29:59.687549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:29:59.687549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:29:59.687549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:29:59.687549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:29:59.687549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:29:59.687549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:29:59.687549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:29:59.687549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:29:59.687549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:29:59.687549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:29:59.687549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:29:59.687549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:29:59.689275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb22e3-7354-41c9-967f-fd14177d3825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:59.690104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb22e3-7354-41c9-967f-fd14177d3825 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:59.690531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb22e3-7354-41c9-967f-fd14177d3825 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:59.690974 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb22e3-7354-41c9-967f-fd14177d3825 None admin] PATH_INFO: `/v3/projects/ee70c279239f4e719fad35784db1a619` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:59.692303 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb22e3-7354-41c9-967f-fd14177d3825 None admin] RBAC: Authorizing `identity:get_project(project_id=ee70c279239f4e719fad35784db1a619)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:59.693913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb22e3-7354-41c9-967f-fd14177d3825 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:59.696593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 217/416] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:29:59 2022] GET /identity/v3/projects/ee70c279239f4e719fad35784db1a619 => generated 344 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:59.697944 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 200/417] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:29:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:29:59.705086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb22e3-7354-41c9-967f-fd14177d3825 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:59.708996 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f09437-1af5-4f37-8a76-f964200eb545 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:59.754773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:29:59.842615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb22e3-7354-41c9-967f-fd14177d3825 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:59.882191 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3053295-0c63-4f76-9147-bb4c8e44f737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:59.882860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3053295-0c63-4f76-9147-bb4c8e44f737 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:59.883136 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3053295-0c63-4f76-9147-bb4c8e44f737 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:59.883457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3053295-0c63-4f76-9147-bb4c8e44f737 None admin] PATH_INFO: `/v3/projects/ee70c279239f4e719fad35784db1a619/groups/3e97f57f232f42a3b1a7d172dae1513b/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:59.886809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3053295-0c63-4f76-9147-bb4c8e44f737 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee70c279239f4e719fad35784db1a619, group_id=3e97f57f232f42a3b1a7d172dae1513b, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:59.888242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3053295-0c63-4f76-9147-bb4c8e44f737 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:59.907082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f09437-1af5-4f37-8a76-f964200eb545 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:29:59.952039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd30fdc-4056-4556-840b-5318d113ce71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:29:59.952039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd30fdc-4056-4556-840b-5318d113ce71 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:29:59.952039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd30fdc-4056-4556-840b-5318d113ce71 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:29:59.952039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd30fdc-4056-4556-840b-5318d113ce71 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:29:59.952462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd30fdc-4056-4556-840b-5318d113ce71 None admin] RBAC: Adding query filter params (name=member) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:29:59.952462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd30fdc-4056-4556-840b-5318d113ce71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:29:59.952462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd30fdc-4056-4556-840b-5318d113ce71 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:29:59.959211 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 201/418] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:29:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:29:59.966590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd30fdc-4056-4556-840b-5318d113ce71 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:29:59.978446 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 218/419] 10.222.0.57 () {58 vars in 1442 bytes} [Tue Aug 9 18:29:59 2022] PUT /identity/v3/projects/ee70c279239f4e719fad35784db1a619/groups/3e97f57f232f42a3b1a7d172dae1513b/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:30:00.080434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:00.096248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd30fdc-4056-4556-840b-5318d113ce71 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:00.148170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f3da-4911-4242-b63b-c99a8e14fb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:00.148800 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f3da-4911-4242-b63b-c99a8e14fb1b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:00.149118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f3da-4911-4242-b63b-c99a8e14fb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:00.149393 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f3da-4911-4242-b63b-c99a8e14fb1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:00.238241 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649f3da-4911-4242-b63b-c99a8e14fb1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:00.239688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649f3da-4911-4242-b63b-c99a8e14fb1b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:00.334699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:00.351035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 202/420] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:29:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:00.357226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3053295-0c63-4f76-9147-bb4c8e44f737 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:00.377069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:00.462471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3053295-0c63-4f76-9147-bb4c8e44f737 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:00.498002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f781c-fcde-4e42-bdc9-ee81529aba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:00.498565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f781c-fcde-4e42-bdc9-ee81529aba1b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:00.498849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f781c-fcde-4e42-bdc9-ee81529aba1b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:00.499126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f781c-fcde-4e42-bdc9-ee81529aba1b None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:00.500259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f781c-fcde-4e42-bdc9-ee81529aba1b None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:00.501803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f781c-fcde-4e42-bdc9-ee81529aba1b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:00.503582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 219/421] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:30:00 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:00.509130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f3da-4911-4242-b63b-c99a8e14fb1b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:00.613429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f3da-4911-4242-b63b-c99a8e14fb1b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:00.649511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329fe36-4f35-442b-920d-500ace0ebb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:00.650171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329fe36-4f35-442b-920d-500ace0ebb08 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:00.650453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329fe36-4f35-442b-920d-500ace0ebb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:00.650730 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329fe36-4f35-442b-920d-500ace0ebb08 None admin] PATH_INFO: `/v3/groups/c336ec08b15c4352a280415e31a2fee8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:00.651862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329fe36-4f35-442b-920d-500ace0ebb08 None admin] RBAC: Authorizing `identity:get_group(group_id=c336ec08b15c4352a280415e31a2fee8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:00.653350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329fe36-4f35-442b-920d-500ace0ebb08 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:00.655287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 203/422] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:30:00 2022] GET /identity/v3/groups/c336ec08b15c4352a280415e31a2fee8 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:00.661044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f781c-fcde-4e42-bdc9-ee81529aba1b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:00.756906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:00.773504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f781c-fcde-4e42-bdc9-ee81529aba1b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:00.799786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4f753-487f-404c-8051-1229c221a129 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:00.800090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4f753-487f-404c-8051-1229c221a129 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:00.800336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4f753-487f-404c-8051-1229c221a129 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:00.801700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 204/423] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:30:00.810816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943bc85-baf2-45ab-8382-f827713b1b76 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:00.811211 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943bc85-baf2-45ab-8382-f827713b1b76 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:00.811598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943bc85-baf2-45ab-8382-f827713b1b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:00.813952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c5a19-9b75-47ab-9d74-bee8f0da2825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:00.813952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c5a19-9b75-47ab-9d74-bee8f0da2825 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:00.813952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c5a19-9b75-47ab-9d74-bee8f0da2825 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:00.813952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c5a19-9b75-47ab-9d74-bee8f0da2825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:00.814514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c5a19-9b75-47ab-9d74-bee8f0da2825 None admin] RBAC: Adding query filter params (group.id=c336ec08b15c4352a280415e31a2fee8, role.id=08a7c861d84b43d7a4da431f31f3c2d7, scope.project.id=5d18db22ee924d578a2f7079502940eb) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:00.814816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c5a19-9b75-47ab-9d74-bee8f0da2825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:00.816250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c5a19-9b75-47ab-9d74-bee8f0da2825 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:00.823219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 220/424] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 9 18:30:00 2022] GET /identity/v3/role_assignments?group.id=c336ec08b15c4352a280415e31a2fee8&scope.project.id=5d18db22ee924d578a2f7079502940eb&role.id=08a7c861d84b43d7a4da431f31f3c2d7 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:00.862056 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6943bc85-baf2-45ab-8382-f827713b1b76 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:00.863292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6943bc85-baf2-45ab-8382-f827713b1b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:00.944367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:00.972120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 205/425] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:30:01.008128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c5a19-9b75-47ab-9d74-bee8f0da2825 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:01.117016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cddc3c-9594-4d7e-9955-df41ad1f9ac1 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:01.117360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cddc3c-9594-4d7e-9955-df41ad1f9ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:01.117613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cddc3c-9594-4d7e-9955-df41ad1f9ac1 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:01.119634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 206/426] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:30:01.119968 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c5a19-9b75-47ab-9d74-bee8f0da2825 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:01.125561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075b4ad-498b-4f7a-8667-d840d9f00f1a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:01.125858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075b4ad-498b-4f7a-8667-d840d9f00f1a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:01.126140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075b4ad-498b-4f7a-8667-d840d9f00f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:01.147233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5075b4ad-498b-4f7a-8667-d840d9f00f1a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:01.148453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5075b4ad-498b-4f7a-8667-d840d9f00f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:01.159769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deafd3-d3fc-4623-8483-9b9a2a0dc015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:01.160788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deafd3-d3fc-4623-8483-9b9a2a0dc015 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:01.161355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deafd3-d3fc-4623-8483-9b9a2a0dc015 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:01.161701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deafd3-d3fc-4623-8483-9b9a2a0dc015 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:01.175031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77deafd3-d3fc-4623-8483-9b9a2a0dc015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-77deafd3-d3fc-4623-8483-9b9a2a0dc015 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:01.182221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:01.183094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:01.183863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:01.183863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:01.183863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:01.183863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:01.183863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:01.183863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:01.183863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:01.183863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:01.183863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:01.183863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:01.183863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 18:30:01.183863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:01.184456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 221/427] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:01.191780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deafd3-d3fc-4623-8483-9b9a2a0dc015 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:01.207704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:01.235991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 207/428] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:01.275126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075b4ad-498b-4f7a-8667-d840d9f00f1a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:01.323524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deafd3-d3fc-4623-8483-9b9a2a0dc015 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:01.375548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd06da-9431-41d7-bb6e-a92350edcfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:01.377025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd06da-9431-41d7-bb6e-a92350edcfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:01.377648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd06da-9431-41d7-bb6e-a92350edcfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:01.378136 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd06da-9431-41d7-bb6e-a92350edcfe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:01.379029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd06da-9431-41d7-bb6e-a92350edcfe2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:01.379540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd06da-9431-41d7-bb6e-a92350edcfe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:01.381170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd06da-9431-41d7-bb6e-a92350edcfe2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:01.393989 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 222/429] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:30:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:01.403539 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075b4ad-498b-4f7a-8667-d840d9f00f1a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:01.403726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd06da-9431-41d7-bb6e-a92350edcfe2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:01.424164 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:01.453129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0343e-2722-4e1d-bcd8-c91c4f2de53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:01.453860 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0343e-2722-4e1d-bcd8-c91c4f2de53d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:01.454143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0343e-2722-4e1d-bcd8-c91c4f2de53d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:01.454415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0343e-2722-4e1d-bcd8-c91c4f2de53d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:01.466123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0343e-2722-4e1d-bcd8-c91c4f2de53d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-85c0343e-2722-4e1d-bcd8-c91c4f2de53d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:01.471590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:01.472484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:01.473275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 208/430] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:30:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:01.481225 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0343e-2722-4e1d-bcd8-c91c4f2de53d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:01.500687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:01.540180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd06da-9431-41d7-bb6e-a92350edcfe2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:01.579819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242aa24-9d21-4098-a868-a125e0ecb3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:01.580445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242aa24-9d21-4098-a868-a125e0ecb3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:01.580722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242aa24-9d21-4098-a868-a125e0ecb3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:01.580991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242aa24-9d21-4098-a868-a125e0ecb3f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:01.615872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0343e-2722-4e1d-bcd8-c91c4f2de53d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:01.659756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3059a-6636-4d20-9393-be1b20ab8ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:01.660458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3059a-6636-4d20-9393-be1b20ab8ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:01.660754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3059a-6636-4d20-9393-be1b20ab8ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:01.661010 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3059a-6636-4d20-9393-be1b20ab8ce5 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:01.661712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3059a-6636-4d20-9393-be1b20ab8ce5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:01.662020 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3059a-6636-4d20-9393-be1b20ab8ce5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:01.663391 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3059a-6636-4d20-9393-be1b20ab8ce5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:01.674326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 209/431] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:01.680158 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3059a-6636-4d20-9393-be1b20ab8ce5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:01.702242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242aa24-9d21-4098-a868-a125e0ecb3f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:01.704117 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242aa24-9d21-4098-a868-a125e0ecb3f4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:01.722795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:01.795756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:01.801623 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3059a-6636-4d20-9393-be1b20ab8ce5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:01.831208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 223/432] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:01.832491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a396115-3784-4830-91fb-5ee7dfcc5342 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:01.832780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a396115-3784-4830-91fb-5ee7dfcc5342 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:01.833056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a396115-3784-4830-91fb-5ee7dfcc5342 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:01.839676 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 224/433] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:01.840492 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a396115-3784-4830-91fb-5ee7dfcc5342 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:01.851589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed7f42-51dd-41e4-a454-6b19ee3a2b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:01.853386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed7f42-51dd-41e4-a454-6b19ee3a2b4b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:01.854035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed7f42-51dd-41e4-a454-6b19ee3a2b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:01.854595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed7f42-51dd-41e4-a454-6b19ee3a2b4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:01.927029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed7f42-51dd-41e4-a454-6b19ee3a2b4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:01.928475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed7f42-51dd-41e4-a454-6b19ee3a2b4b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:01.949113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a396115-3784-4830-91fb-5ee7dfcc5342 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:01.962868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:01.987146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f09cd-cdad-480f-ac0b-91ba4a808541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:01.988543 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f09cd-cdad-480f-ac0b-91ba4a808541 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:01.989018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f09cd-cdad-480f-ac0b-91ba4a808541 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:01.989366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f09cd-cdad-480f-ac0b-91ba4a808541 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:01.990591 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f09cd-cdad-480f-ac0b-91ba4a808541 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:01.992211 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f09cd-cdad-480f-ac0b-91ba4a808541 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:01.998458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 225/434] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:30:01 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:01.998458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46ccbb-e363-48f9-afc8-3548f1f5f5de None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:01.998458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46ccbb-e363-48f9-afc8-3548f1f5f5de None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:01.998458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46ccbb-e363-48f9-afc8-3548f1f5f5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:02.013717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-9f46ccbb-e363-48f9-afc8-3548f1f5f5de None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:02.014909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9f46ccbb-e363-48f9-afc8-3548f1f5f5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:02.043406 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 210/435] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:02.043406 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed7f42-51dd-41e4-a454-6b19ee3a2b4b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:02.052378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:02.091269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 226/436] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:02.092370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46ccbb-e363-48f9-afc8-3548f1f5f5de None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:02.158449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed7f42-51dd-41e4-a454-6b19ee3a2b4b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:02.196801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e63ad5-1fb2-4176-a7c3-87ad3916a461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:02.197512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e63ad5-1fb2-4176-a7c3-87ad3916a461 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:02.197630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46ccbb-e363-48f9-afc8-3548f1f5f5de None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:02.197928 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e63ad5-1fb2-4176-a7c3-87ad3916a461 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:02.198269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e63ad5-1fb2-4176-a7c3-87ad3916a461 None admin] PATH_INFO: `/v3/groups/c336ec08b15c4352a280415e31a2fee8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:02.199520 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e63ad5-1fb2-4176-a7c3-87ad3916a461 None admin] RBAC: Authorizing `identity:get_group(group_id=c336ec08b15c4352a280415e31a2fee8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:02.201122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e63ad5-1fb2-4176-a7c3-87ad3916a461 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:02.207657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 211/437] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:30:02 2022] GET /identity/v3/groups/c336ec08b15c4352a280415e31a2fee8 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:02.207657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e63ad5-1fb2-4176-a7c3-87ad3916a461 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:02.208851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:02.236529 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b53029-86a0-4ca4-99e4-dfd61361bcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:02.237144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b53029-86a0-4ca4-99e4-dfd61361bcae None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:02.237429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b53029-86a0-4ca4-99e4-dfd61361bcae None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:02.237737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b53029-86a0-4ca4-99e4-dfd61361bcae None admin] PATH_INFO: `/v3/projects/5d18db22ee924d578a2f7079502940eb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:02.238925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b53029-86a0-4ca4-99e4-dfd61361bcae None admin] RBAC: Authorizing `identity:get_project(project_id=5d18db22ee924d578a2f7079502940eb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:02.240460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b53029-86a0-4ca4-99e4-dfd61361bcae None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:02.246996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 227/438] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:30:02 2022] GET /identity/v3/projects/5d18db22ee924d578a2f7079502940eb => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:02.246996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b53029-86a0-4ca4-99e4-dfd61361bcae None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:02.291799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:02.324599 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e63ad5-1fb2-4176-a7c3-87ad3916a461 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:02.362135 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5cd58-be8f-4ce3-b41d-9d135dba34ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:02.362778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5cd58-be8f-4ce3-b41d-9d135dba34ac None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:02.363066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5cd58-be8f-4ce3-b41d-9d135dba34ac None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:02.363336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5cd58-be8f-4ce3-b41d-9d135dba34ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:02.375991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b53029-86a0-4ca4-99e4-dfd61361bcae None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:02.393303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5cd58-be8f-4ce3-b41d-9d135dba34ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-2bb5cd58-be8f-4ce3-b41d-9d135dba34ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:02.398569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:02.400163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 212/439] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:30:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:02.401561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5cd58-be8f-4ce3-b41d-9d135dba34ac None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:02.420957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559ecd3-152b-482d-8522-0a16fef5de64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:02.421590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559ecd3-152b-482d-8522-0a16fef5de64 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:02.421870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559ecd3-152b-482d-8522-0a16fef5de64 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:02.422221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559ecd3-152b-482d-8522-0a16fef5de64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:02.422883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559ecd3-152b-482d-8522-0a16fef5de64 None admin] RBAC: Adding query filter params (group.id=c336ec08b15c4352a280415e31a2fee8, role.id=08a7c861d84b43d7a4da431f31f3c2d7, scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:02.423187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559ecd3-152b-482d-8522-0a16fef5de64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:02.424625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559ecd3-152b-482d-8522-0a16fef5de64 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:02.431572 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 228/440] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 9 18:30:02 2022] GET /identity/v3/role_assignments?group.id=c336ec08b15c4352a280415e31a2fee8&scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85&role.id=08a7c861d84b43d7a4da431f31f3c2d7 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:02.443712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559ecd3-152b-482d-8522-0a16fef5de64 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:02.474857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:02.520841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5cd58-be8f-4ce3-b41d-9d135dba34ac None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:02.564972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14870b1f-74bf-4201-b898-99e4822de91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:02.566366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14870b1f-74bf-4201-b898-99e4822de91a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:02.566366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14870b1f-74bf-4201-b898-99e4822de91a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:02.566366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14870b1f-74bf-4201-b898-99e4822de91a None admin] PATH_INFO: `/v3/groups/c336ec08b15c4352a280415e31a2fee8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:02.569055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14870b1f-74bf-4201-b898-99e4822de91a None admin] RBAC: Authorizing `identity:get_group(group_id=c336ec08b15c4352a280415e31a2fee8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:02.570241 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14870b1f-74bf-4201-b898-99e4822de91a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:02.572086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 213/441] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:30:02 2022] GET /identity/v3/groups/c336ec08b15c4352a280415e31a2fee8 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:02.573304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df54186-bd18-4203-b917-8fd242a18886 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:02.573944 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df54186-bd18-4203-b917-8fd242a18886 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:02.573944 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df54186-bd18-4203-b917-8fd242a18886 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:02.575329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 214/442] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:02.579350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df54186-bd18-4203-b917-8fd242a18886 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:02.598294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:02.601642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559ecd3-152b-482d-8522-0a16fef5de64 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:02.640645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bc175-c018-4c98-b7eb-de74afa85aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:02.641413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bc175-c018-4c98-b7eb-de74afa85aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:02.641781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bc175-c018-4c98-b7eb-de74afa85aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:02.642058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bc175-c018-4c98-b7eb-de74afa85aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:02.643568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bc175-c018-4c98-b7eb-de74afa85aa3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:02.643568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bc175-c018-4c98-b7eb-de74afa85aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:02.644736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bc175-c018-4c98-b7eb-de74afa85aa3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:02.653946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 229/443] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:02.655212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed9fbd-0f16-4255-8d55-30d64e8f407e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:02.655571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed9fbd-0f16-4255-8d55-30d64e8f407e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:02.655855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed9fbd-0f16-4255-8d55-30d64e8f407e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:02.673727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b3ed9fbd-0f16-4255-8d55-30d64e8f407e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:02.674915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b3ed9fbd-0f16-4255-8d55-30d64e8f407e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:02.711995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df54186-bd18-4203-b917-8fd242a18886 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:02.744819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 230/444] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:02.745705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed9fbd-0f16-4255-8d55-30d64e8f407e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:02.749243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:02.759415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5f20c-5274-4184-8f6f-98eb5320faf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:02.760119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5f20c-5274-4184-8f6f-98eb5320faf2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:02.760867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5f20c-5274-4184-8f6f-98eb5320faf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:02.761651 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5f20c-5274-4184-8f6f-98eb5320faf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:02.762530 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5f20c-5274-4184-8f6f-98eb5320faf2 None admin] RBAC: Adding query filter params (group.id=c336ec08b15c4352a280415e31a2fee8, role.id=1d1b5b96118040f9a4fd019bbcd93071, scope.project.id=5d18db22ee924d578a2f7079502940eb) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:02.762916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5f20c-5274-4184-8f6f-98eb5320faf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:02.765222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5f20c-5274-4184-8f6f-98eb5320faf2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:02.773876 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 215/445] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 9 18:30:02 2022] GET /identity/v3/role_assignments?group.id=c336ec08b15c4352a280415e31a2fee8&scope.project.id=5d18db22ee924d578a2f7079502940eb&role.id=1d1b5b96118040f9a4fd019bbcd93071 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:30:02.773876 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5f20c-5274-4184-8f6f-98eb5320faf2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:02.860579 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:02.869999 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed9fbd-0f16-4255-8d55-30d64e8f407e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:02.891761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5f20c-5274-4184-8f6f-98eb5320faf2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:02.908119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b58d54-daf4-478a-87ad-73893b550d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:02.908840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58d54-daf4-478a-87ad-73893b550d60 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:02.909235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58d54-daf4-478a-87ad-73893b550d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:02.909605 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58d54-daf4-478a-87ad-73893b550d60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:02.936356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3b150-18da-4d78-bf0e-ab3606ad7d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:02.936356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3b150-18da-4d78-bf0e-ab3606ad7d7d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:02.936356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3b150-18da-4d78-bf0e-ab3606ad7d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:02.936356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3b150-18da-4d78-bf0e-ab3606ad7d7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:02.945143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3b150-18da-4d78-bf0e-ab3606ad7d7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-35a3b150-18da-4d78-bf0e-ab3606ad7d7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:02.950964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:02.951865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:02.952632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 216/446] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:02.958224 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3b150-18da-4d78-bf0e-ab3606ad7d7d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:03.006521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b58d54-daf4-478a-87ad-73893b550d60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:03.007998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b58d54-daf4-478a-87ad-73893b550d60 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:03.034853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:03.110471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3b150-18da-4d78-bf0e-ab3606ad7d7d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:03.148051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:03.156863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3806a-8c5d-4039-9677-ef91b80f4f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:03.157479 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3806a-8c5d-4039-9677-ef91b80f4f60 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:03.157792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3806a-8c5d-4039-9677-ef91b80f4f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:03.158067 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3806a-8c5d-4039-9677-ef91b80f4f60 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:03.163060 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3806a-8c5d-4039-9677-ef91b80f4f60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:03.163060 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3806a-8c5d-4039-9677-ef91b80f4f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:03.163243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 231/447] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:03.163316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3806a-8c5d-4039-9677-ef91b80f4f60 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:03.165343 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b58d54-daf4-478a-87ad-73893b550d60 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:03.174721 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 217/448] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:03.180523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3806a-8c5d-4039-9677-ef91b80f4f60 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:03.311230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3806a-8c5d-4039-9677-ef91b80f4f60 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:03.315575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b58d54-daf4-478a-87ad-73893b550d60 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:03.333490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:03.351516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0c2c7-e1ad-4a6c-89ae-131b4cd07c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:03.353163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0c2c7-e1ad-4a6c-89ae-131b4cd07c75 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:03.353163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0c2c7-e1ad-4a6c-89ae-131b4cd07c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:03.353163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0c2c7-e1ad-4a6c-89ae-131b4cd07c75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:03.358482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcc12b-129e-4919-80b8-3fae21f5aa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:03.369555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcc12b-129e-4919-80b8-3fae21f5aa44 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:03.369555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcc12b-129e-4919-80b8-3fae21f5aa44 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:03.369555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcc12b-129e-4919-80b8-3fae21f5aa44 None admin] PATH_INFO: `/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:03.369555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcc12b-129e-4919-80b8-3fae21f5aa44 None admin] RBAC: Authorizing `identity:get_project(project_id=38ed82c9c6da41e596c28a3fb28c8c85)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:03.369555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcc12b-129e-4919-80b8-3fae21f5aa44 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:03.369555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 232/449] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:30:03 2022] GET /identity/v3/projects/38ed82c9c6da41e596c28a3fb28c8c85 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:03.370440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcc12b-129e-4919-80b8-3fae21f5aa44 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:03.431080 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0c2c7-e1ad-4a6c-89ae-131b4cd07c75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:03.432614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0c2c7-e1ad-4a6c-89ae-131b4cd07c75 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:03.435146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:03.480745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcc12b-129e-4919-80b8-3fae21f5aa44 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:03.519625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a1fde-2f42-45bf-ac81-6a836cc962ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:03.520267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a1fde-2f42-45bf-ac81-6a836cc962ad None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:03.520547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a1fde-2f42-45bf-ac81-6a836cc962ad None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:03.520831 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a1fde-2f42-45bf-ac81-6a836cc962ad None admin] PATH_INFO: `/v3/groups/c336ec08b15c4352a280415e31a2fee8` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:03.521990 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a1fde-2f42-45bf-ac81-6a836cc962ad None admin] RBAC: Authorizing `identity:get_group(group_id=c336ec08b15c4352a280415e31a2fee8)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:03.523436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a1fde-2f42-45bf-ac81-6a836cc962ad None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:03.525010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 233/450] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:30:03 2022] GET /identity/v3/groups/c336ec08b15c4352a280415e31a2fee8 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:30:03.530728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a1fde-2f42-45bf-ac81-6a836cc962ad None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:03.535732 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 218/451] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:03.541420 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0c2c7-e1ad-4a6c-89ae-131b4cd07c75 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:03.560226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:03.641206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a1fde-2f42-45bf-ac81-6a836cc962ad None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:03.651594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0c2c7-e1ad-4a6c-89ae-131b4cd07c75 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:03.670437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:03.678605 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea83a74-29e7-447a-bb65-17c8df30d522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:03.679477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea83a74-29e7-447a-bb65-17c8df30d522 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:03.679757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea83a74-29e7-447a-bb65-17c8df30d522 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:03.680029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea83a74-29e7-447a-bb65-17c8df30d522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:03.683044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea83a74-29e7-447a-bb65-17c8df30d522 None admin] RBAC: Adding query filter params (group.id=c336ec08b15c4352a280415e31a2fee8, role.id=1d1b5b96118040f9a4fd019bbcd93071, scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:03.683324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea83a74-29e7-447a-bb65-17c8df30d522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:03.687119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea83a74-29e7-447a-bb65-17c8df30d522 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:03.694595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 234/452] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 9 18:30:03 2022] GET /identity/v3/role_assignments?group.id=c336ec08b15c4352a280415e31a2fee8&scope.project.id=38ed82c9c6da41e596c28a3fb28c8c85&role.id=1d1b5b96118040f9a4fd019bbcd93071 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:30:03.701040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f0c51-855d-4093-82e7-c5f9f6e5545a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:03.701727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f0c51-855d-4093-82e7-c5f9f6e5545a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:03.702005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f0c51-855d-4093-82e7-c5f9f6e5545a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:03.702269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f0c51-855d-4093-82e7-c5f9f6e5545a None admin] PATH_INFO: `/v3/projects/ee70c279239f4e719fad35784db1a619` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:03.703448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f0c51-855d-4093-82e7-c5f9f6e5545a None admin] RBAC: Authorizing `identity:get_project(project_id=ee70c279239f4e719fad35784db1a619)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:03.704924 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f0c51-855d-4093-82e7-c5f9f6e5545a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:03.706655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 219/453] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:30:03 2022] GET /identity/v3/projects/ee70c279239f4e719fad35784db1a619 => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:03.712769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea83a74-29e7-447a-bb65-17c8df30d522 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:03.824909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea83a74-29e7-447a-bb65-17c8df30d522 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:03.860019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf03d9-2324-4609-aff0-4c8a2ed4e44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:03.860665 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf03d9-2324-4609-aff0-4c8a2ed4e44f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:03.861991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf03d9-2324-4609-aff0-4c8a2ed4e44f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:03.861991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf03d9-2324-4609-aff0-4c8a2ed4e44f None admin] PATH_INFO: `/v3/groups/3e97f57f232f42a3b1a7d172dae1513b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:03.862331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf03d9-2324-4609-aff0-4c8a2ed4e44f None admin] RBAC: Authorizing `identity:get_group(group_id=3e97f57f232f42a3b1a7d172dae1513b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:03.863772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf03d9-2324-4609-aff0-4c8a2ed4e44f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:03.865335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 235/454] 10.222.0.57 () {56 vars in 1180 bytes} [Tue Aug 9 18:30:03 2022] GET /identity/v3/groups/3e97f57f232f42a3b1a7d172dae1513b => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:03.871835 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f0c51-855d-4093-82e7-c5f9f6e5545a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:03.960179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:03.974648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f0c51-855d-4093-82e7-c5f9f6e5545a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:04.010895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a40956-6494-4441-aadd-c661e24b8843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:04.011496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a40956-6494-4441-aadd-c661e24b8843 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:04.011781 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a40956-6494-4441-aadd-c661e24b8843 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:04.012080 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a40956-6494-4441-aadd-c661e24b8843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:04.012849 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a40956-6494-4441-aadd-c661e24b8843 None admin] RBAC: Adding query filter params (group.id=3e97f57f232f42a3b1a7d172dae1513b, role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=ee70c279239f4e719fad35784db1a619) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:04.013162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a40956-6494-4441-aadd-c661e24b8843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:04.014633 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a40956-6494-4441-aadd-c661e24b8843 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:04.020684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 220/455] 10.222.0.57 () {56 vars in 1376 bytes} [Tue Aug 9 18:30:03 2022] GET /identity/v3/role_assignments?group.id=3e97f57f232f42a3b1a7d172dae1513b&scope.project.id=ee70c279239f4e719fad35784db1a619&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:30:06.436526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0e50e-6ed6-4277-a6c2-f4296d455032 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:06.436526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0e50e-6ed6-4277-a6c2-f4296d455032 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:06.436526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0e50e-6ed6-4277-a6c2-f4296d455032 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:06.437594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 236/456] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:06.442787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287c559-816c-494d-8f5d-092389dbc2c1 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:06.443192 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287c559-816c-494d-8f5d-092389dbc2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:06.443374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287c559-816c-494d-8f5d-092389dbc2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:06.469574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4287c559-816c-494d-8f5d-092389dbc2c1 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:06.471110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4287c559-816c-494d-8f5d-092389dbc2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:06.615716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 221/457] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:06.651795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0e50e-6ed6-4277-a6c2-f4296d455032 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:06.775894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1b429-afb3-43c3-aa48-735fab7217b6 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:06.776240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1b429-afb3-43c3-aa48-735fab7217b6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:06.776415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1b429-afb3-43c3-aa48-735fab7217b6 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:06.781655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 222/458] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:06.784054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d03286-5d4e-491f-82c6-afe6778abdb2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:06.784337 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d03286-5d4e-491f-82c6-afe6778abdb2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:06.784621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d03286-5d4e-491f-82c6-afe6778abdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:06.810055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:06.815460 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-61d03286-5d4e-491f-82c6-afe6778abdb2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:06.816662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-61d03286-5d4e-491f-82c6-afe6778abdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:06.888605 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0e50e-6ed6-4277-a6c2-f4296d455032 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:06.966858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c76a18-7221-49a1-be39-dfe1563031de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:06.967507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c76a18-7221-49a1-be39-dfe1563031de None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:06.967791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c76a18-7221-49a1-be39-dfe1563031de None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:06.968062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c76a18-7221-49a1-be39-dfe1563031de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:06.979206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 223/459] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:30:06.988743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c76a18-7221-49a1-be39-dfe1563031de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:06.990297 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c76a18-7221-49a1-be39-dfe1563031de None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-a2c76a18-7221-49a1-be39-dfe1563031de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:07.007963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:07.009453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:07.010349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:07.010349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:07.010349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:07.014105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 237/460] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:07.019630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d03286-5d4e-491f-82c6-afe6778abdb2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:07.037968 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:07.043718 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c76a18-7221-49a1-be39-dfe1563031de None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:07.157931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c76a18-7221-49a1-be39-dfe1563031de None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:07.164323 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d03286-5d4e-491f-82c6-afe6778abdb2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:07.173960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:07.199514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3980ba0-87bc-42a9-a29d-fa5760d1e159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:07.200264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3980ba0-87bc-42a9-a29d-fa5760d1e159 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:07.200545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3980ba0-87bc-42a9-a29d-fa5760d1e159 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:07.200843 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3980ba0-87bc-42a9-a29d-fa5760d1e159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:07.206737 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636ddfe-de9a-4e40-8521-d4f2332319bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:07.207449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636ddfe-de9a-4e40-8521-d4f2332319bd None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:07.208027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636ddfe-de9a-4e40-8521-d4f2332319bd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:07.208398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636ddfe-de9a-4e40-8521-d4f2332319bd None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:07.209138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636ddfe-de9a-4e40-8521-d4f2332319bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:07.209548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636ddfe-de9a-4e40-8521-d4f2332319bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:07.211021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636ddfe-de9a-4e40-8521-d4f2332319bd None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:07.213380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3980ba0-87bc-42a9-a29d-fa5760d1e159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:07.215443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3980ba0-87bc-42a9-a29d-fa5760d1e159 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:07.223881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 224/461] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:07.225318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da95966-42ad-4c3e-bd42-8173f4275f7f None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:07.225730 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da95966-42ad-4c3e-bd42-8173f4275f7f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:07.226065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da95966-42ad-4c3e-bd42-8173f4275f7f None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:07.227628 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 225/462] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:07.228767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da95966-42ad-4c3e-bd42-8173f4275f7f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f3980ba0-87bc-42a9-a29d-fa5760d1e159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:07.237715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:07.238885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:07.239914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:07.239914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:07.239914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:07.241644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 238/463] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:07.243441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644fa7a-dec4-47c5-8f8f-ee6df88a3acc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:07.243814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644fa7a-dec4-47c5-8f8f-ee6df88a3acc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:07.244258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644fa7a-dec4-47c5-8f8f-ee6df88a3acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:07.262583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-9644fa7a-dec4-47c5-8f8f-ee6df88a3acc None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:07.263903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9644fa7a-dec4-47c5-8f8f-ee6df88a3acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:07.341548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 239/464] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:07.341736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:07.341814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644fa7a-dec4-47c5-8f8f-ee6df88a3acc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:07.378798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da95966-42ad-4c3e-bd42-8173f4275f7f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:07.413873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7ac74-aa6b-433e-9128-4090366392d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:07.414954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7ac74-aa6b-433e-9128-4090366392d8 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:07.415203 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7ac74-aa6b-433e-9128-4090366392d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:07.415471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7ac74-aa6b-433e-9128-4090366392d8 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:07.416189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7ac74-aa6b-433e-9128-4090366392d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:07.417637 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7ac74-aa6b-433e-9128-4090366392d8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:07.435723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:07.442189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644fa7a-dec4-47c5-8f8f-ee6df88a3acc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:07.474628 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c5900-1359-422e-903b-118653619fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:07.475261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c5900-1359-422e-903b-118653619fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:07.475539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c5900-1359-422e-903b-118653619fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:07.475805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c5900-1359-422e-903b-118653619fb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:07.476447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c5900-1359-422e-903b-118653619fb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:07.476769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c5900-1359-422e-903b-118653619fb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:07.478688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c5900-1359-422e-903b-118653619fb6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:07.489818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 240/465] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:07.489818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b9f07-8ff6-4bde-bdd0-72f4fec23772 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:07.489818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b9f07-8ff6-4bde-bdd0-72f4fec23772 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:07.489818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b9f07-8ff6-4bde-bdd0-72f4fec23772 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:07.489818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 241/466] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:07.489818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b9f07-8ff6-4bde-bdd0-72f4fec23772 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:07.508947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 226/467] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:30:07 2022] POST /identity/v3/users => generated 246 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:07.510232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7ac74-aa6b-433e-9128-4090366392d8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:07.628666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b9f07-8ff6-4bde-bdd0-72f4fec23772 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:07.667604 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7ac74-aa6b-433e-9128-4090366392d8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:07.675337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0d614-da8b-4cb3-817e-c86175915efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:07.675975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0d614-da8b-4cb3-817e-c86175915efb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:07.676253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0d614-da8b-4cb3-817e-c86175915efb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:07.676520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0d614-da8b-4cb3-817e-c86175915efb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:07.690990 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0d614-da8b-4cb3-817e-c86175915efb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:07.692723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0d614-da8b-4cb3-817e-c86175915efb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:07.706478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-68f0d614-da8b-4cb3-817e-c86175915efb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:07.710298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:07.711141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:07.712399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:07.712399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:07.712399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:07.712399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 242/468] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:07.712907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110d5d3-7c74-455a-8cdc-c99ece657121 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:07.712907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110d5d3-7c74-455a-8cdc-c99ece657121 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:07.713157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110d5d3-7c74-455a-8cdc-c99ece657121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:07.724940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915a5fb-8db9-4655-bc02-9b122292ba40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:07.725921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915a5fb-8db9-4655-bc02-9b122292ba40 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:07.726359 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915a5fb-8db9-4655-bc02-9b122292ba40 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:07.728162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915a5fb-8db9-4655-bc02-9b122292ba40 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:07.728162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915a5fb-8db9-4655-bc02-9b122292ba40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:07.730017 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915a5fb-8db9-4655-bc02-9b122292ba40 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:07.748326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:07.754946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a110d5d3-7c74-455a-8cdc-c99ece657121 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:07.756126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a110d5d3-7c74-455a-8cdc-c99ece657121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:07.821558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 227/469] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:30:07 2022] POST /identity/v3/users => generated 248 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:07.822668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915a5fb-8db9-4655-bc02-9b122292ba40 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:07.843651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 243/470] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:07.873071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110d5d3-7c74-455a-8cdc-c99ece657121 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:08.026679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915a5fb-8db9-4655-bc02-9b122292ba40 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:08.038602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110d5d3-7c74-455a-8cdc-c99ece657121 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:08.068146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd102a-06f9-44cd-9bc3-2346f7aac04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:08.068768 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd102a-06f9-44cd-9bc3-2346f7aac04c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:08.069051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd102a-06f9-44cd-9bc3-2346f7aac04c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:08.069323 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd102a-06f9-44cd-9bc3-2346f7aac04c None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:08.070005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd102a-06f9-44cd-9bc3-2346f7aac04c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:08.070312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd102a-06f9-44cd-9bc3-2346f7aac04c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:08.071916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd102a-06f9-44cd-9bc3-2346f7aac04c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:08.087715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 228/471] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:08.089179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd2a6d-382e-4f1e-b981-ac36aec262c2 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:08.089461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd2a6d-382e-4f1e-b981-ac36aec262c2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:08.089760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd2a6d-382e-4f1e-b981-ac36aec262c2 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:08.091267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 229/472] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:08.092955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd2a6d-382e-4f1e-b981-ac36aec262c2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:08.135352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e333a-a26a-40ad-b882-2d6eee1f1805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:08.136464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e333a-a26a-40ad-b882-2d6eee1f1805 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:08.136946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e333a-a26a-40ad-b882-2d6eee1f1805 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:08.137415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e333a-a26a-40ad-b882-2d6eee1f1805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:08.140589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:08.146761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:08.151534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e333a-a26a-40ad-b882-2d6eee1f1805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:08.153246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e333a-a26a-40ad-b882-2d6eee1f1805 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-6a7e333a-a26a-40ad-b882-2d6eee1f1805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:08.171194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:08.172135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:08.172974 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:08.172974 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:08.172974 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:08.175123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 244/473] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:30:08.181378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e9e1b-3e3f-4504-8043-2a8d9fb32e28 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:08.181969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e9e1b-3e3f-4504-8043-2a8d9fb32e28 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:08.182453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e9e1b-3e3f-4504-8043-2a8d9fb32e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:08.201559 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-201e9e1b-3e3f-4504-8043-2a8d9fb32e28 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:08.204753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-201e9e1b-3e3f-4504-8043-2a8d9fb32e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:08.209625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd2a6d-382e-4f1e-b981-ac36aec262c2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:08.248653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f28b02-1a1b-40dd-a3b8-f3268ba2903d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:08.249259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f28b02-1a1b-40dd-a3b8-f3268ba2903d None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:08.249561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f28b02-1a1b-40dd-a3b8-f3268ba2903d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:08.249868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f28b02-1a1b-40dd-a3b8-f3268ba2903d None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:08.250575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f28b02-1a1b-40dd-a3b8-f3268ba2903d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:08.252202 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f28b02-1a1b-40dd-a3b8-f3268ba2903d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:08.293077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 245/474] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:08.293077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e9e1b-3e3f-4504-8043-2a8d9fb32e28 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:08.349062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 230/475] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:30:08 2022] POST /identity/v3/users => generated 248 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:30:08.351141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f28b02-1a1b-40dd-a3b8-f3268ba2903d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:08.465901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e9e1b-3e3f-4504-8043-2a8d9fb32e28 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:08.521125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:08.538099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:08.544448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f28b02-1a1b-40dd-a3b8-f3268ba2903d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:08.546171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68386e2-a6e4-4b3d-8423-d3472a160bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:08.546764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68386e2-a6e4-4b3d-8423-d3472a160bfe None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:08.547045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68386e2-a6e4-4b3d-8423-d3472a160bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:08.547318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68386e2-a6e4-4b3d-8423-d3472a160bfe None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:08.547977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68386e2-a6e4-4b3d-8423-d3472a160bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:08.548923 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68386e2-a6e4-4b3d-8423-d3472a160bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:08.549838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68386e2-a6e4-4b3d-8423-d3472a160bfe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:08.563876 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 246/476] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:08.569358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68386e2-a6e4-4b3d-8423-d3472a160bfe None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:08.589730 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890ec99-f9b4-48ec-80ba-c1102b8bb6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:08.590788 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890ec99-f9b4-48ec-80ba-c1102b8bb6be None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:08.591268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890ec99-f9b4-48ec-80ba-c1102b8bb6be None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:08.591734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890ec99-f9b4-48ec-80ba-c1102b8bb6be None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:08.592586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890ec99-f9b4-48ec-80ba-c1102b8bb6be None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:08.594236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890ec99-f9b4-48ec-80ba-c1102b8bb6be None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:08.626787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.sql.core [None req-1890ec99-f9b4-48ec-80ba-c1102b8bb6be None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 18:30:08.626787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 18:30:08.626787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [parameters: {'id': 'fcefebf59f06426c8f730d8c6ac12c2f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 18:30:08.626787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80445) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-1890ec99-f9b4-48ec-80ba-c1102b8bb6be None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application conn.query(q) Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result.read() Aug 09 18:30:08.650632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return ref.to_dict() Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 18:30:08.651578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application next(self.gen) Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application yield resource Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application next(self.gen) Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise self.value Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application session.commit() Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 18:30:08.652510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.session.flush() Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application compat.raise_( Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 18:30:08.653364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 18:30:08.655581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application conn.query(q) Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application result.read() Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 18:30:08.658566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [parameters: {'id': 'fcefebf59f06426c8f730d8c6ac12c2f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:08.660591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 18:30:08.667573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:08.670045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 231/477] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Aug 9 18:30:08 2022] POST /identity/v3/roles => generated 144 bytes in 305 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:08.674248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890ec99-f9b4-48ec-80ba-c1102b8bb6be None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:08.699981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68386e2-a6e4-4b3d-8423-d3472a160bfe None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:08.762246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c335be-4798-46ec-8b0b-b18747e18415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:08.762861 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c335be-4798-46ec-8b0b-b18747e18415 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:08.763137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c335be-4798-46ec-8b0b-b18747e18415 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:08.763402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c335be-4798-46ec-8b0b-b18747e18415 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:08.764051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c335be-4798-46ec-8b0b-b18747e18415 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:08.765501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c335be-4798-46ec-8b0b-b18747e18415 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:08.795975 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890ec99-f9b4-48ec-80ba-c1102b8bb6be None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:08.833449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd85b3-7789-47de-a3d6-4ae607b9f40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:08.834106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd85b3-7789-47de-a3d6-4ae607b9f40b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:08.834388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd85b3-7789-47de-a3d6-4ae607b9f40b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:08.834660 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd85b3-7789-47de-a3d6-4ae607b9f40b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:08.845896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd85b3-7789-47de-a3d6-4ae607b9f40b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-fadd85b3-7789-47de-a3d6-4ae607b9f40b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:08.851026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:08.852502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:08.853287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 232/478] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:30:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:08.862781 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd85b3-7789-47de-a3d6-4ae607b9f40b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:08.867089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 247/479] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:30:08 2022] POST /identity/v3/users => generated 249 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:08.912205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:09.019083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd85b3-7789-47de-a3d6-4ae607b9f40b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:09.072227 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd1bd2-ac27-43fd-9ff8-5c9b99d2b8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:09.072844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd1bd2-ac27-43fd-9ff8-5c9b99d2b8dc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:09.073127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd1bd2-ac27-43fd-9ff8-5c9b99d2b8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:09.073401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd1bd2-ac27-43fd-9ff8-5c9b99d2b8dc None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:09.074110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd1bd2-ac27-43fd-9ff8-5c9b99d2b8dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:09.074417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd1bd2-ac27-43fd-9ff8-5c9b99d2b8dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:09.075818 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd1bd2-ac27-43fd-9ff8-5c9b99d2b8dc None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:09.085482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 233/480] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:09.884906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2d027-1eb3-4d0e-8db1-125896148c1c None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:09.885429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2d027-1eb3-4d0e-8db1-125896148c1c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:09.885560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2d027-1eb3-4d0e-8db1-125896148c1c None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:09.887122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 248/481] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:09.892156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec0033-ac92-43ad-94e5-bfb9e33f7ea0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:09.892580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec0033-ac92-43ad-94e5-bfb9e33f7ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:09.892747 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec0033-ac92-43ad-94e5-bfb9e33f7ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:09.911369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-82ec0033-ac92-43ad-94e5-bfb9e33f7ea0 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:09.912599 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-82ec0033-ac92-43ad-94e5-bfb9e33f7ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:09.986533 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 234/482] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:10.035290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2d027-1eb3-4d0e-8db1-125896148c1c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:10.080060 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:10.174824 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2d027-1eb3-4d0e-8db1-125896148c1c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:10.224168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404b498-6f1d-4509-ac7b-31f53c0f1516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:10.224805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404b498-6f1d-4509-ac7b-31f53c0f1516 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:10.225086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404b498-6f1d-4509-ac7b-31f53c0f1516 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:10.225358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404b498-6f1d-4509-ac7b-31f53c0f1516 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:10.246361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404b498-6f1d-4509-ac7b-31f53c0f1516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-5404b498-6f1d-4509-ac7b-31f53c0f1516 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:10.251113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:10.251971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:10.252764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 249/483] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:10.259441 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec0033-ac92-43ad-94e5-bfb9e33f7ea0 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:10.296561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:10.305625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51275af6-97da-4e46-aa52-08a094f3cad0 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:10.311684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51275af6-97da-4e46-aa52-08a094f3cad0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:10.311684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51275af6-97da-4e46-aa52-08a094f3cad0 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:10.311684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 250/484] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:10.312267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356e2b-5dad-405d-b0a2-57ac2928e6a4 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:10.312553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356e2b-5dad-405d-b0a2-57ac2928e6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:10.312827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356e2b-5dad-405d-b0a2-57ac2928e6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:10.341167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-20356e2b-5dad-405d-b0a2-57ac2928e6a4 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:10.342397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-20356e2b-5dad-405d-b0a2-57ac2928e6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:10.415672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec0033-ac92-43ad-94e5-bfb9e33f7ea0 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:10.459141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529f7d1-9806-4edf-8279-d766aad35ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:10.462098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529f7d1-9806-4edf-8279-d766aad35ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:10.462598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529f7d1-9806-4edf-8279-d766aad35ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:10.463963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529f7d1-9806-4edf-8279-d766aad35ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:10.464837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529f7d1-9806-4edf-8279-d766aad35ff0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:10.466717 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529f7d1-9806-4edf-8279-d766aad35ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:10.469365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529f7d1-9806-4edf-8279-d766aad35ff0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:10.490826 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 235/485] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:10.498398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529f7d1-9806-4edf-8279-d766aad35ff0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:10.526774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 251/486] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:10.573013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20356e2b-5dad-405d-b0a2-57ac2928e6a4 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:10.624839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:10.659241 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:10.694693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529f7d1-9806-4edf-8279-d766aad35ff0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:10.735551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bec3f2-94d0-43d0-95d5-081c702ee672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:10.736518 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bec3f2-94d0-43d0-95d5-081c702ee672 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:10.736998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bec3f2-94d0-43d0-95d5-081c702ee672 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:10.737485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bec3f2-94d0-43d0-95d5-081c702ee672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:10.762903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bec3f2-94d0-43d0-95d5-081c702ee672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:10.766035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bec3f2-94d0-43d0-95d5-081c702ee672 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:10.786556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20356e2b-5dad-405d-b0a2-57ac2928e6a4 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-e8bec3f2-94d0-43d0-95d5-081c702ee672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:10.794697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:10.795613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:10.796465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:10.796465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:10.796465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:10.798958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 236/487] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:10.804640 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bec3f2-94d0-43d0-95d5-081c702ee672 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:10.870133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20ff01-17f7-4190-b985-fe7e3d5abd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:10.870794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ff01-17f7-4190-b985-fe7e3d5abd5a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:10.871077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ff01-17f7-4190-b985-fe7e3d5abd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:10.871519 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ff01-17f7-4190-b985-fe7e3d5abd5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:10.895224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20ff01-17f7-4190-b985-fe7e3d5abd5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-bc20ff01-17f7-4190-b985-fe7e3d5abd5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:10.900175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:10.901070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:10.901902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:10.901902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:10.901902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:10.901902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:10.901902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:10.901902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:10.901902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:10.901902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:10.901902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:10.901902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:10.901902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:10.901902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:10.910232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 252/488] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:30:10.915869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20ff01-17f7-4190-b985-fe7e3d5abd5a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:10.938719 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bec3f2-94d0-43d0-95d5-081c702ee672 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:10.969980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:10.979680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7d026-4524-403b-a497-ae30963d78fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:10.980715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7d026-4524-403b-a497-ae30963d78fb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:10.981264 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7d026-4524-403b-a497-ae30963d78fb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:10.981816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7d026-4524-403b-a497-ae30963d78fb None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:10.982805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7d026-4524-403b-a497-ae30963d78fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:10.983318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7d026-4524-403b-a497-ae30963d78fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:10.985148 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7d026-4524-403b-a497-ae30963d78fb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:11.001436 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 237/489] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:30:11.010797 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7d026-4524-403b-a497-ae30963d78fb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:11.045213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:11.075350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20ff01-17f7-4190-b985-fe7e3d5abd5a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:11.147289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d70aec-adee-42de-9047-7a3ab888d8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:11.147919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d70aec-adee-42de-9047-7a3ab888d8bf None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.148468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d70aec-adee-42de-9047-7a3ab888d8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.148468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d70aec-adee-42de-9047-7a3ab888d8bf None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.149167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d70aec-adee-42de-9047-7a3ab888d8bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:11.149471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d70aec-adee-42de-9047-7a3ab888d8bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:11.150901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d70aec-adee-42de-9047-7a3ab888d8bf None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:11.152505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7d026-4524-403b-a497-ae30963d78fb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:11.169490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 253/490] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:30:11.174900 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d70aec-adee-42de-9047-7a3ab888d8bf None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:11.193565 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929998fa-60c0-416a-b635-5a010e7dc12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:11.194799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929998fa-60c0-416a-b635-5a010e7dc12e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.195332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929998fa-60c0-416a-b635-5a010e7dc12e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.195960 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929998fa-60c0-416a-b635-5a010e7dc12e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.204956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929998fa-60c0-416a-b635-5a010e7dc12e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:11.206826 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929998fa-60c0-416a-b635-5a010e7dc12e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-929998fa-60c0-416a-b635-5a010e7dc12e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:11.222281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:11.223249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:11.224223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:11.224223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:11.224223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:11.224223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:11.224223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:11.224223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:11.224223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:11.224223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:11.224223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:30:11.224223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:11.229470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 238/491] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 9 18:30:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:30:11.236184 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929998fa-60c0-416a-b635-5a010e7dc12e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:11.353867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:11.365212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d70aec-adee-42de-9047-7a3ab888d8bf None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:11.385613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929998fa-60c0-416a-b635-5a010e7dc12e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:11.415100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b834d1-fc1b-4a4b-827f-c6ffa2aad0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:11.415740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b834d1-fc1b-4a4b-827f-c6ffa2aad0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.416022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b834d1-fc1b-4a4b-827f-c6ffa2aad0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.416293 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b834d1-fc1b-4a4b-827f-c6ffa2aad0e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.431795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b834d1-fc1b-4a4b-827f-c6ffa2aad0e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:11.432837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbcaae-00c6-4d70-82a7-a82bd2a6b6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:11.433262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b834d1-fc1b-4a4b-827f-c6ffa2aad0e8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:11.433706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbcaae-00c6-4d70-82a7-a82bd2a6b6ee None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.434089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbcaae-00c6-4d70-82a7-a82bd2a6b6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.434472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbcaae-00c6-4d70-82a7-a82bd2a6b6ee None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.435313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbcaae-00c6-4d70-82a7-a82bd2a6b6ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:11.435988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbcaae-00c6-4d70-82a7-a82bd2a6b6ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:11.437954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbcaae-00c6-4d70-82a7-a82bd2a6b6ee None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:11.441904 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-30b834d1-fc1b-4a4b-827f-c6ffa2aad0e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:11.450356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:11.461432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:11.462307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:11.462307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:11.462307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:11.462307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 254/492] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:11.462307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd4eff-8cbf-4138-a311-ab79cddd85d8 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.462307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd4eff-8cbf-4138-a311-ab79cddd85d8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.462307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd4eff-8cbf-4138-a311-ab79cddd85d8 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.462307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 255/493] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:11.462307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe48ee-ecf5-4864-9839-1146665039a9 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.462791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe48ee-ecf5-4864-9839-1146665039a9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.462791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe48ee-ecf5-4864-9839-1146665039a9 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.462791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 256/494] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:11.462791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe48ee-ecf5-4864-9839-1146665039a9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:11.487457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 239/495] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 9 18:30:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:30:11.488594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156911a-2f6b-4cbd-95b6-7a1ab08cd42e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.488880 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156911a-2f6b-4cbd-95b6-7a1ab08cd42e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.489153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156911a-2f6b-4cbd-95b6-7a1ab08cd42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.507431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f156911a-2f6b-4cbd-95b6-7a1ab08cd42e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:11.508654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f156911a-2f6b-4cbd-95b6-7a1ab08cd42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:11.575570 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe48ee-ecf5-4864-9839-1146665039a9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:11.585558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 240/496] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:11.585558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf8e39-77cc-4728-9d90-614de884cb3d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.585558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf8e39-77cc-4728-9d90-614de884cb3d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.585558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf8e39-77cc-4728-9d90-614de884cb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.599850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ccdf8e39-77cc-4728-9d90-614de884cb3d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:11.601213 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ccdf8e39-77cc-4728-9d90-614de884cb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:11.615791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1f066-2b2c-48a5-b4b5-c3f5946b2086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:11.616398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1f066-2b2c-48a5-b4b5-c3f5946b2086 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.616678 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1f066-2b2c-48a5-b4b5-c3f5946b2086 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.616948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1f066-2b2c-48a5-b4b5-c3f5946b2086 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.617677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1f066-2b2c-48a5-b4b5-c3f5946b2086 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:11.617982 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1f066-2b2c-48a5-b4b5-c3f5946b2086 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:11.619485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1f066-2b2c-48a5-b4b5-c3f5946b2086 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:11.631330 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 257/497] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:11.632017 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:11.634456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1f066-2b2c-48a5-b4b5-c3f5946b2086 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:11.672397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 241/498] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:11.674469 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211798a2-2801-4d7c-b0ae-9d6b0519a441 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.674469 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211798a2-2801-4d7c-b0ae-9d6b0519a441 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.674469 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211798a2-2801-4d7c-b0ae-9d6b0519a441 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.675844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 242/499] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:11.676543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211798a2-2801-4d7c-b0ae-9d6b0519a441 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:11.702817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:11.744802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1f066-2b2c-48a5-b4b5-c3f5946b2086 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:11.783743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211798a2-2801-4d7c-b0ae-9d6b0519a441 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:11.784658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b5b96-b6bb-4e0f-9aea-aa2ec40bb1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:11.785266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b5b96-b6bb-4e0f-9aea-aa2ec40bb1ae None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.785559 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b5b96-b6bb-4e0f-9aea-aa2ec40bb1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.785832 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b5b96-b6bb-4e0f-9aea-aa2ec40bb1ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.818704 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ac00a-7f5a-44cb-a3a5-9d90fc547b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:11.819331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ac00a-7f5a-44cb-a3a5-9d90fc547b06 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.819609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ac00a-7f5a-44cb-a3a5-9d90fc547b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.819894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ac00a-7f5a-44cb-a3a5-9d90fc547b06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.829936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ac00a-7f5a-44cb-a3a5-9d90fc547b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-5f8ac00a-7f5a-44cb-a3a5-9d90fc547b06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:11.834622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:11.835503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:11.836793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 243/500] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:11.837435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ac00a-7f5a-44cb-a3a5-9d90fc547b06 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:11.859718 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b5b96-b6bb-4e0f-9aea-aa2ec40bb1ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:11.861164 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b5b96-b6bb-4e0f-9aea-aa2ec40bb1ae None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:11.867522 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:11.917179 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:11.938823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ac00a-7f5a-44cb-a3a5-9d90fc547b06 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:11.965706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 258/501] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:11.965706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4e198-3820-4917-84fb-827cdfb45546 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.965706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4e198-3820-4917-84fb-827cdfb45546 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.965706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4e198-3820-4917-84fb-827cdfb45546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.981093 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d05387-9d72-4a99-9917-d4f4b0daa795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:11.981850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d05387-9d72-4a99-9917-d4f4b0daa795 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:11.982136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d05387-9d72-4a99-9917-d4f4b0daa795 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:11.982408 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d05387-9d72-4a99-9917-d4f4b0daa795 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:11.982500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4fb4e198-3820-4917-84fb-827cdfb45546 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:11.983856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4fb4e198-3820-4917-84fb-827cdfb45546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:11.989957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d05387-9d72-4a99-9917-d4f4b0daa795 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:11.991420 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d05387-9d72-4a99-9917-d4f4b0daa795 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-f7d05387-9d72-4a99-9917-d4f4b0daa795 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:12.006153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:12.007119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:12.008516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:12.008516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:12.008516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:12.008516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:12.008516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:12.008516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:12.008516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:12.008516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:12.008516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 18:30:12.008516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:12.008516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 244/502] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:12.009082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d05387-9d72-4a99-9917-d4f4b0daa795 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:12.071430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 259/503] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:12.072284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4e198-3820-4917-84fb-827cdfb45546 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:12.108027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:12.122986 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d05387-9d72-4a99-9917-d4f4b0daa795 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:12.167046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a299a-38c9-480b-b6b5-4d744d7659d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:12.167671 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a299a-38c9-480b-b6b5-4d744d7659d3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:12.167945 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a299a-38c9-480b-b6b5-4d744d7659d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:12.168214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a299a-38c9-480b-b6b5-4d744d7659d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:12.178345 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a299a-38c9-480b-b6b5-4d744d7659d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:12.182857 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-ae2a299a-38c9-480b-b6b5-4d744d7659d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:12.183305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:12.184325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:12.185143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 245/504] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:12.185825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a299a-38c9-480b-b6b5-4d744d7659d3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:12.198221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4e198-3820-4917-84fb-827cdfb45546 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:12.233637 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ff56a-b15c-4fb7-8dd2-7b88230ca613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:12.234246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ff56a-b15c-4fb7-8dd2-7b88230ca613 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:12.234525 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ff56a-b15c-4fb7-8dd2-7b88230ca613 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:12.234804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ff56a-b15c-4fb7-8dd2-7b88230ca613 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:12.235464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ff56a-b15c-4fb7-8dd2-7b88230ca613 None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:12.235767 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ff56a-b15c-4fb7-8dd2-7b88230ca613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:12.237148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ff56a-b15c-4fb7-8dd2-7b88230ca613 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:12.244997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 260/505] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:12.245809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ff56a-b15c-4fb7-8dd2-7b88230ca613 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:12.300229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a299a-38c9-480b-b6b5-4d744d7659d3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:12.355834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b3245-5456-4a61-93f5-059e2b1521ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:12.356853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b3245-5456-4a61-93f5-059e2b1521ec None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:12.357137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b3245-5456-4a61-93f5-059e2b1521ec None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:12.357550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b3245-5456-4a61-93f5-059e2b1521ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:12.369530 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ff56a-b15c-4fb7-8dd2-7b88230ca613 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:12.369696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b3245-5456-4a61-93f5-059e2b1521ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:12.371132 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b3245-5456-4a61-93f5-059e2b1521ec None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:12.379741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-db2b3245-5456-4a61-93f5-059e2b1521ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:12.389402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:12.395933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:12.396785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:12.396785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:12.396785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:12.396785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 246/506] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:12.396785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b3245-5456-4a61-93f5-059e2b1521ec None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:12.416265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470ffea-6e80-4d8e-9ffd-3794051b8ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:12.416879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470ffea-6e80-4d8e-9ffd-3794051b8ceb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:12.417161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470ffea-6e80-4d8e-9ffd-3794051b8ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:12.417689 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470ffea-6e80-4d8e-9ffd-3794051b8ceb None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:12.418663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470ffea-6e80-4d8e-9ffd-3794051b8ceb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:12.418663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470ffea-6e80-4d8e-9ffd-3794051b8ceb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:12.420108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470ffea-6e80-4d8e-9ffd-3794051b8ceb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:12.458597 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 261/507] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Aug 9 18:30:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:12.458597 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470ffea-6e80-4d8e-9ffd-3794051b8ceb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:12.471232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:12.502015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b3245-5456-4a61-93f5-059e2b1521ec None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:12.541097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36895647-975d-4fc7-95f9-4ed2abf5811a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:12.541755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36895647-975d-4fc7-95f9-4ed2abf5811a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:12.542037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36895647-975d-4fc7-95f9-4ed2abf5811a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:12.542313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36895647-975d-4fc7-95f9-4ed2abf5811a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:12.555330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36895647-975d-4fc7-95f9-4ed2abf5811a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:12.557019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36895647-975d-4fc7-95f9-4ed2abf5811a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-36895647-975d-4fc7-95f9-4ed2abf5811a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:12.573245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:12.574328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:12.575168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:12.575168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:12.575168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:12.575328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470ffea-6e80-4d8e-9ffd-3794051b8ceb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:12.575399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 247/508] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:12.576088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36895647-975d-4fc7-95f9-4ed2abf5811a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:12.621085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f0127-d127-4aae-906d-b82b91ab1b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:12.621672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f0127-d127-4aae-906d-b82b91ab1b9a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:12.621955 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f0127-d127-4aae-906d-b82b91ab1b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:12.622226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f0127-d127-4aae-906d-b82b91ab1b9a None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:12.622894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f0127-d127-4aae-906d-b82b91ab1b9a None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:12.623191 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f0127-d127-4aae-906d-b82b91ab1b9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:12.624575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f0127-d127-4aae-906d-b82b91ab1b9a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:12.632847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:12.636135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 262/509] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:12.637548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f0127-d127-4aae-906d-b82b91ab1b9a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:12.683767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36895647-975d-4fc7-95f9-4ed2abf5811a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:12.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:12.716173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e03ed-6e02-4d5f-9ef3-2e2bd64ca51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:12.716832 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e03ed-6e02-4d5f-9ef3-2e2bd64ca51e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:12.717131 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e03ed-6e02-4d5f-9ef3-2e2bd64ca51e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:12.717403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e03ed-6e02-4d5f-9ef3-2e2bd64ca51e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:12.730472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e03ed-6e02-4d5f-9ef3-2e2bd64ca51e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:12.731980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e03ed-6e02-4d5f-9ef3-2e2bd64ca51e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:12.742334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f0127-d127-4aae-906d-b82b91ab1b9a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-567e03ed-6e02-4d5f-9ef3-2e2bd64ca51e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:12.746631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:12.747694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:12.753419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:12.753419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:12.753419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:12.753419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 248/510] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:12.753419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e03ed-6e02-4d5f-9ef3-2e2bd64ca51e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:12.779007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c958e-3708-4e8c-a28f-2048d74ca665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:12.779639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c958e-3708-4e8c-a28f-2048d74ca665 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:12.779919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c958e-3708-4e8c-a28f-2048d74ca665 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:12.780190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c958e-3708-4e8c-a28f-2048d74ca665 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:12.780830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c958e-3708-4e8c-a28f-2048d74ca665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:12.781138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c958e-3708-4e8c-a28f-2048d74ca665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:12.782763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c958e-3708-4e8c-a28f-2048d74ca665 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:12.794667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 263/511] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:12.795570 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c958e-3708-4e8c-a28f-2048d74ca665 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:12.846262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e03ed-6e02-4d5f-9ef3-2e2bd64ca51e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:12.847777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:12.880778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d4f1d-fd25-452b-8b89-094ba8f188db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:12.881281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d4f1d-fd25-452b-8b89-094ba8f188db None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:12.881572 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d4f1d-fd25-452b-8b89-094ba8f188db None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:12.881849 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d4f1d-fd25-452b-8b89-094ba8f188db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:12.903448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c958e-3708-4e8c-a28f-2048d74ca665 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:12.934856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:12.941182 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318328f9-733e-4f58-a83a-554ad2660c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:12.941818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318328f9-733e-4f58-a83a-554ad2660c2d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:12.942093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318328f9-733e-4f58-a83a-554ad2660c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:12.942364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318328f9-733e-4f58-a83a-554ad2660c2d None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:12.943006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318328f9-733e-4f58-a83a-554ad2660c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:12.943307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318328f9-733e-4f58-a83a-554ad2660c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:12.944697 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318328f9-733e-4f58-a83a-554ad2660c2d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:12.951780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d4f1d-fd25-452b-8b89-094ba8f188db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:12.953243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d4f1d-fd25-452b-8b89-094ba8f188db None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:12.957576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 264/512] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:12.957576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318328f9-733e-4f58-a83a-554ad2660c2d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:13.054553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 249/513] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:13.055955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d4f1d-fd25-452b-8b89-094ba8f188db None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:13.058259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318328f9-733e-4f58-a83a-554ad2660c2d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:13.061685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:13.092408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07618b9-e394-4ae0-9f0c-1219c5f7679a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:13.093025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07618b9-e394-4ae0-9f0c-1219c5f7679a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:13.093293 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07618b9-e394-4ae0-9f0c-1219c5f7679a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:13.093633 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07618b9-e394-4ae0-9f0c-1219c5f7679a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:13.105180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07618b9-e394-4ae0-9f0c-1219c5f7679a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:13.106663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07618b9-e394-4ae0-9f0c-1219c5f7679a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-a07618b9-e394-4ae0-9f0c-1219c5f7679a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:13.120481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:13.121483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:13.127025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:13.127025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:13.127025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:13.127025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 265/514] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:13.127025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07618b9-e394-4ae0-9f0c-1219c5f7679a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:13.160217 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d4f1d-fd25-452b-8b89-094ba8f188db None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:13.171607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:13.193488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6fc2f-ca77-45c7-8ca8-f4418a772181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:13.194116 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6fc2f-ca77-45c7-8ca8-f4418a772181 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:13.194395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6fc2f-ca77-45c7-8ca8-f4418a772181 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:13.194666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6fc2f-ca77-45c7-8ca8-f4418a772181 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:13.195305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6fc2f-ca77-45c7-8ca8-f4418a772181 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:13.195609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6fc2f-ca77-45c7-8ca8-f4418a772181 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:13.196977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6fc2f-ca77-45c7-8ca8-f4418a772181 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:13.207897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 250/515] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:13.208669 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6fc2f-ca77-45c7-8ca8-f4418a772181 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:13.228325 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07618b9-e394-4ae0-9f0c-1219c5f7679a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:13.265838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6806b84-5384-42c5-88ff-f155cfd6f9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:13.266471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6806b84-5384-42c5-88ff-f155cfd6f9cd None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:13.266750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6806b84-5384-42c5-88ff-f155cfd6f9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:13.267018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6806b84-5384-42c5-88ff-f155cfd6f9cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:13.274357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:13.274626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6806b84-5384-42c5-88ff-f155cfd6f9cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:13.276081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6806b84-5384-42c5-88ff-f155cfd6f9cd None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-c6806b84-5384-42c5-88ff-f155cfd6f9cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:13.287614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:13.288640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:13.290190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:13.290190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:13.290190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:13.290190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:13.290190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:13.290190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:13.290190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:13.290190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:13.290190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 266/516] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:13.290747 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6806b84-5384-42c5-88ff-f155cfd6f9cd None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:13.316370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6fc2f-ca77-45c7-8ca8-f4418a772181 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:13.350921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d387c2-34f3-4d7a-84ea-7c148ca3c7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:13.351522 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d387c2-34f3-4d7a-84ea-7c148ca3c7d0 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:13.352434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d387c2-34f3-4d7a-84ea-7c148ca3c7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:13.352713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d387c2-34f3-4d7a-84ea-7c148ca3c7d0 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:13.353360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d387c2-34f3-4d7a-84ea-7c148ca3c7d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:13.354808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d387c2-34f3-4d7a-84ea-7c148ca3c7d0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:13.389346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6806b84-5384-42c5-88ff-f155cfd6f9cd None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:13.390964 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:13.423891 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b21764-96f3-4135-a34a-8b3893911d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:13.424446 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b21764-96f3-4135-a34a-8b3893911d58 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:13.424724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b21764-96f3-4135-a34a-8b3893911d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:13.424991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b21764-96f3-4135-a34a-8b3893911d58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:13.434821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b21764-96f3-4135-a34a-8b3893911d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:13.436267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b21764-96f3-4135-a34a-8b3893911d58 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-52b21764-96f3-4135-a34a-8b3893911d58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:13.450704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:13.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:13.452669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:13.452669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:13.452669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:13.452669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 267/517] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:13.452669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b21764-96f3-4135-a34a-8b3893911d58 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:13.460616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 251/518] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:30:13 2022] POST /identity/v3/users => generated 247 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:13.462170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d387c2-34f3-4d7a-84ea-7c148ca3c7d0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:13.574838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b21764-96f3-4135-a34a-8b3893911d58 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:13.581090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d387c2-34f3-4d7a-84ea-7c148ca3c7d0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:13.600663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:13.618375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96d2bd-fa1b-4629-ba14-58c42210fd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:13.618581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c9b25-c99d-42aa-b554-90cd117f2f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:13.633405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96d2bd-fa1b-4629-ba14-58c42210fd35 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:13.633604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c9b25-c99d-42aa-b554-90cd117f2f64 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:13.633686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96d2bd-fa1b-4629-ba14-58c42210fd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:13.633756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c9b25-c99d-42aa-b554-90cd117f2f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:13.633835 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96d2bd-fa1b-4629-ba14-58c42210fd35 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:13.633912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c9b25-c99d-42aa-b554-90cd117f2f64 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:13.633912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c9b25-c99d-42aa-b554-90cd117f2f64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:13.633912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c9b25-c99d-42aa-b554-90cd117f2f64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:13.633912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c9b25-c99d-42aa-b554-90cd117f2f64 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:13.634141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96d2bd-fa1b-4629-ba14-58c42210fd35 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:13.634141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96d2bd-fa1b-4629-ba14-58c42210fd35 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:13.634766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 268/519] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:13.636650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c9b25-c99d-42aa-b554-90cd117f2f64 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-4b96d2bd-fa1b-4629-ba14-58c42210fd35 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:13.640461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:13.641470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:13.642855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:13.642855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:13.642855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:13.642855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:13.642855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:13.642855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:13.642855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:13.642855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:13.642855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 18:30:13.642855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:13.642855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 252/520] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:13.642855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96d2bd-fa1b-4629-ba14-58c42210fd35 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:13.654532 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:13.761731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c9b25-c99d-42aa-b554-90cd117f2f64 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:13.762639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96d2bd-fa1b-4629-ba14-58c42210fd35 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:13.804049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d4319-a66f-4d9f-b87f-98610937c248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:13.805778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d4319-a66f-4d9f-b87f-98610937c248 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:13.806198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d4319-a66f-4d9f-b87f-98610937c248 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:13.806740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d4319-a66f-4d9f-b87f-98610937c248 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:13.807578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d4319-a66f-4d9f-b87f-98610937c248 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:13.808022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d4319-a66f-4d9f-b87f-98610937c248 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:13.809753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d4319-a66f-4d9f-b87f-98610937c248 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:13.811927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcca207-517b-4329-9794-da7072e8c75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:13.813075 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcca207-517b-4329-9794-da7072e8c75f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:13.813729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcca207-517b-4329-9794-da7072e8c75f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:13.814301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcca207-517b-4329-9794-da7072e8c75f None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:13.815249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcca207-517b-4329-9794-da7072e8c75f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:13.815843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcca207-517b-4329-9794-da7072e8c75f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:13.817352 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcca207-517b-4329-9794-da7072e8c75f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:13.828407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 269/521] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:13.829981 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 253/522] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:13.833993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d4319-a66f-4d9f-b87f-98610937c248 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:13.834261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcca207-517b-4329-9794-da7072e8c75f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:13.878681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:13.920937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:13.942507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcca207-517b-4329-9794-da7072e8c75f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:13.946340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d4319-a66f-4d9f-b87f-98610937c248 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:13.979302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dba8f-40ff-45f1-8fe1-a4a9b318c266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:13.980011 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dba8f-40ff-45f1-8fe1-a4a9b318c266 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:13.980236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dba8f-40ff-45f1-8fe1-a4a9b318c266 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:13.980484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dba8f-40ff-45f1-8fe1-a4a9b318c266 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:13.981150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46dba8f-40ff-45f1-8fe1-a4a9b318c266 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:13.981451 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46dba8f-40ff-45f1-8fe1-a4a9b318c266 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:13.982058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a3eef-6e48-475d-af63-74dea314e291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:13.982687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a3eef-6e48-475d-af63-74dea314e291 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:13.982896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46dba8f-40ff-45f1-8fe1-a4a9b318c266 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:13.983003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a3eef-6e48-475d-af63-74dea314e291 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:13.983224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a3eef-6e48-475d-af63-74dea314e291 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:13.991157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a3eef-6e48-475d-af63-74dea314e291 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:13.992623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a3eef-6e48-475d-af63-74dea314e291 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:14.006516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 254/523] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Aug 9 18:30:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:14.006516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dba8f-40ff-45f1-8fe1-a4a9b318c266 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-7b5a3eef-6e48-475d-af63-74dea314e291 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:14.006778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:14.007594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:14.008630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:14.008630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:14.008630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:14.008630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:14.008630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:14.008630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:14.008630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:14.008630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:14.008630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 18:30:14.008630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:14.008630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 270/524] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:14.008630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a3eef-6e48-475d-af63-74dea314e291 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:14.117720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a3eef-6e48-475d-af63-74dea314e291 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:14.119025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:14.128995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dba8f-40ff-45f1-8fe1-a4a9b318c266 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:14.158269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699292d4-719e-4201-a6d1-3511931f4d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:14.159157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699292d4-719e-4201-a6d1-3511931f4d53 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:14.159581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699292d4-719e-4201-a6d1-3511931f4d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:14.159852 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699292d4-719e-4201-a6d1-3511931f4d53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:14.161757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:14.165709 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff8977-1fff-4fda-8257-234310f5814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:14.166309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff8977-1fff-4fda-8257-234310f5814e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:14.166588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff8977-1fff-4fda-8257-234310f5814e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:14.166860 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff8977-1fff-4fda-8257-234310f5814e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:14.167592 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff8977-1fff-4fda-8257-234310f5814e None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=e2ecd7d1cce644b0982cc23740537933) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:14.167904 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff8977-1fff-4fda-8257-234310f5814e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:14.168590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699292d4-719e-4201-a6d1-3511931f4d53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:14.169341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff8977-1fff-4fda-8257-234310f5814e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:14.170430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699292d4-719e-4201-a6d1-3511931f4d53 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:14.175218 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 255/525] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:14 2022] GET /identity/v3/role_assignments?user.id=e2ecd7d1cce644b0982cc23740537933&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-699292d4-719e-4201-a6d1-3511931f4d53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:14.182493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:14.183483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:14.184544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:14.184544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:14.184544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:14.184544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:14.184544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:14.184544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:14.184544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:14.184544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:14.184960 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff8977-1fff-4fda-8257-234310f5814e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:14.186566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 271/526] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:14.187747 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699292d4-719e-4201-a6d1-3511931f4d53 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:14.343307 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff8977-1fff-4fda-8257-234310f5814e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:14.347646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699292d4-719e-4201-a6d1-3511931f4d53 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:14.381872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972937bd-fec4-4a9b-bef5-87c0a5987868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:14.382394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972937bd-fec4-4a9b-bef5-87c0a5987868 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:14.382699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972937bd-fec4-4a9b-bef5-87c0a5987868 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:14.382976 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972937bd-fec4-4a9b-bef5-87c0a5987868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:14.394926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b4d89-a2cb-464a-945d-72ad6d626d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:14.396303 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b4d89-a2cb-464a-945d-72ad6d626d45 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:14.397071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b4d89-a2cb-464a-945d-72ad6d626d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:14.397586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b4d89-a2cb-464a-945d-72ad6d626d45 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:14.398791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b4d89-a2cb-464a-945d-72ad6d626d45 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:14.399749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b4d89-a2cb-464a-945d-72ad6d626d45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:14.401954 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b4d89-a2cb-464a-945d-72ad6d626d45 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:14.420751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:14.424357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 272/527] 10.222.0.57 () {56 vars in 1139 bytes} [Tue Aug 9 18:30:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:14.425512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b4d89-a2cb-464a-945d-72ad6d626d45 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:14.466845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972937bd-fec4-4a9b-bef5-87c0a5987868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:14.468392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972937bd-fec4-4a9b-bef5-87c0a5987868 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:14.502525 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:14.568669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b4d89-a2cb-464a-945d-72ad6d626d45 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:14.587034 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 256/528] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:14.588285 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972937bd-fec4-4a9b-bef5-87c0a5987868 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:14.611694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd535e9-e6ea-4773-b67a-c275a51f8a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:14.612317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd535e9-e6ea-4773-b67a-c275a51f8a39 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:14.612708 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd535e9-e6ea-4773-b67a-c275a51f8a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:14.612988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd535e9-e6ea-4773-b67a-c275a51f8a39 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:14.613730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd535e9-e6ea-4773-b67a-c275a51f8a39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:14.614042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd535e9-e6ea-4773-b67a-c275a51f8a39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:14.615505 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd535e9-e6ea-4773-b67a-c275a51f8a39 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:14.629332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 273/529] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:14.630279 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd535e9-e6ea-4773-b67a-c275a51f8a39 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:14.673631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:14.727283 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972937bd-fec4-4a9b-bef5-87c0a5987868 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:14.759341 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd535e9-e6ea-4773-b67a-c275a51f8a39 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:14.760535 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513cf54-a0fb-4b3d-bdf3-93b4c6cd4681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:14.761154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513cf54-a0fb-4b3d-bdf3-93b4c6cd4681 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:14.761436 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513cf54-a0fb-4b3d-bdf3-93b4c6cd4681 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:14.761750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513cf54-a0fb-4b3d-bdf3-93b4c6cd4681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:14.778727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:14.806643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd62964-981a-4aad-bbcd-a67a2c26431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:14.807996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd62964-981a-4aad-bbcd-a67a2c26431e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:14.808463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd62964-981a-4aad-bbcd-a67a2c26431e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:14.808905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd62964-981a-4aad-bbcd-a67a2c26431e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:14.821540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd62964-981a-4aad-bbcd-a67a2c26431e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:14.823276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd62964-981a-4aad-bbcd-a67a2c26431e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-1bd62964-981a-4aad-bbcd-a67a2c26431e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:14.838859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:14.839765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:14.840597 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:14.840597 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:14.840597 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:14.840768 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513cf54-a0fb-4b3d-bdf3-93b4c6cd4681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:14.840768 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513cf54-a0fb-4b3d-bdf3-93b4c6cd4681 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:14.842778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 274/530] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:14.843738 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd62964-981a-4aad-bbcd-a67a2c26431e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:14.926633 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:14.969901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd62964-981a-4aad-bbcd-a67a2c26431e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:14.992312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 257/531] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:14.995357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513cf54-a0fb-4b3d-bdf3-93b4c6cd4681 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:15.020512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92606c80-c597-40b9-936f-bb9d7efbf85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:15.021149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92606c80-c597-40b9-936f-bb9d7efbf85e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:15.021433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92606c80-c597-40b9-936f-bb9d7efbf85e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:15.021760 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92606c80-c597-40b9-936f-bb9d7efbf85e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:15.022769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92606c80-c597-40b9-936f-bb9d7efbf85e None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=77300e25b9004a598f8f7f12f3cf4866) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:15.024076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92606c80-c597-40b9-936f-bb9d7efbf85e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:15.025885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92606c80-c597-40b9-936f-bb9d7efbf85e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:15.037701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 275/532] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:14 2022] GET /identity/v3/role_assignments?user.id=77300e25b9004a598f8f7f12f3cf4866&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:15.039146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92606c80-c597-40b9-936f-bb9d7efbf85e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:15.105826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:15.177694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513cf54-a0fb-4b3d-bdf3-93b4c6cd4681 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:15.248653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c642b-880d-4010-9b4a-20764f6941b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:15.249745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c642b-880d-4010-9b4a-20764f6941b2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:15.250223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c642b-880d-4010-9b4a-20764f6941b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:15.250686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c642b-880d-4010-9b4a-20764f6941b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:15.251576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c642b-880d-4010-9b4a-20764f6941b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:15.252810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c642b-880d-4010-9b4a-20764f6941b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:15.254509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c642b-880d-4010-9b4a-20764f6941b2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:15.266185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92606c80-c597-40b9-936f-bb9d7efbf85e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:15.269957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 258/533] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:15.278814 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c642b-880d-4010-9b4a-20764f6941b2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:15.303769 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:15.362280 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fb298-0e31-4f2b-9fc0-90ff4d3c2563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:15.362941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fb298-0e31-4f2b-9fc0-90ff4d3c2563 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:15.363226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fb298-0e31-4f2b-9fc0-90ff4d3c2563 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:15.363506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fb298-0e31-4f2b-9fc0-90ff4d3c2563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:15.396265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c642b-880d-4010-9b4a-20764f6941b2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:15.402355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fb298-0e31-4f2b-9fc0-90ff4d3c2563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:15.403823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fb298-0e31-4f2b-9fc0-90ff4d3c2563 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-c19fb298-0e31-4f2b-9fc0-90ff4d3c2563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:15.421504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:15.422482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:15.423308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:15.423308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:15.423308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:15.423308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 276/534] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:15.428669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fb298-0e31-4f2b-9fc0-90ff4d3c2563 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:15.449550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250aa3ac-c9b5-4f88-a60c-cec0f4028668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:15.449550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250aa3ac-c9b5-4f88-a60c-cec0f4028668 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:15.449550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250aa3ac-c9b5-4f88-a60c-cec0f4028668 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:15.449550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250aa3ac-c9b5-4f88-a60c-cec0f4028668 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:15.469551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250aa3ac-c9b5-4f88-a60c-cec0f4028668 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:15.469551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250aa3ac-c9b5-4f88-a60c-cec0f4028668 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-250aa3ac-c9b5-4f88-a60c-cec0f4028668 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:15.488155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:15.489049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:15.489884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:15.489884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:15.489884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:15.489884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:15.489884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:15.489884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:15.489884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:15.489884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:15.493030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 259/535] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:15.496541 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250aa3ac-c9b5-4f88-a60c-cec0f4028668 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:15.553245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:15.576331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fb298-0e31-4f2b-9fc0-90ff4d3c2563 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:15.616940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653666ef-0596-4cb6-bbc9-26799c946290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:15.618465 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653666ef-0596-4cb6-bbc9-26799c946290 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:15.619616 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653666ef-0596-4cb6-bbc9-26799c946290 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:15.620276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653666ef-0596-4cb6-bbc9-26799c946290 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:15.621128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653666ef-0596-4cb6-bbc9-26799c946290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:15.623028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653666ef-0596-4cb6-bbc9-26799c946290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:15.625403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653666ef-0596-4cb6-bbc9-26799c946290 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:15.643014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 277/536] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:15.648300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653666ef-0596-4cb6-bbc9-26799c946290 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:15.691782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:15.721243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250aa3ac-c9b5-4f88-a60c-cec0f4028668 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:15.770306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28e4e9-6607-4bcf-abe1-aa047d4fa753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:15.771444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28e4e9-6607-4bcf-abe1-aa047d4fa753 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:15.771903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28e4e9-6607-4bcf-abe1-aa047d4fa753 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:15.772431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28e4e9-6607-4bcf-abe1-aa047d4fa753 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:15.772951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653666ef-0596-4cb6-bbc9-26799c946290 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:15.773540 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28e4e9-6607-4bcf-abe1-aa047d4fa753 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:15.774045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28e4e9-6607-4bcf-abe1-aa047d4fa753 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:15.776843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28e4e9-6607-4bcf-abe1-aa047d4fa753 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:15.811989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 260/537] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:15.813846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a9c8d-05e6-4a8b-bb60-41c2233fbf6f None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:15.814753 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a9c8d-05e6-4a8b-bb60-41c2233fbf6f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:15.815732 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a9c8d-05e6-4a8b-bb60-41c2233fbf6f None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:15.818123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 261/538] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:15.819135 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a9c8d-05e6-4a8b-bb60-41c2233fbf6f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:15.832701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed551f-3cd9-4ad2-862c-cb855c81c0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:15.833433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed551f-3cd9-4ad2-862c-cb855c81c0ab None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:15.833830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed551f-3cd9-4ad2-862c-cb855c81c0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:15.834178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed551f-3cd9-4ad2-862c-cb855c81c0ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:15.843617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed551f-3cd9-4ad2-862c-cb855c81c0ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:15.845356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed551f-3cd9-4ad2-862c-cb855c81c0ab None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:15.848668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-cbed551f-3cd9-4ad2-862c-cb855c81c0ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:15.858733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:15.866278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:15.867100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:15.867100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:15.867100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:15.867100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:15.867100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:15.867100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:15.867100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:15.867100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:15.867100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 278/539] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:30:15.867100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683ee35-0133-4bf5-9633-95ae80772001 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:15.867100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683ee35-0133-4bf5-9633-95ae80772001 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:15.867694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683ee35-0133-4bf5-9633-95ae80772001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:15.889564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e683ee35-0133-4bf5-9633-95ae80772001 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:15.889564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e683ee35-0133-4bf5-9633-95ae80772001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:15.957482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a9c8d-05e6-4a8b-bb60-41c2233fbf6f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:15.975302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 279/540] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:15.976860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683ee35-0133-4bf5-9633-95ae80772001 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:15.998595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c32c2-61c5-4cfd-aeb6-39c414e87294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:15.999238 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c32c2-61c5-4cfd-aeb6-39c414e87294 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:15.999483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c32c2-61c5-4cfd-aeb6-39c414e87294 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:15.999747 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c32c2-61c5-4cfd-aeb6-39c414e87294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:16.000938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c32c2-61c5-4cfd-aeb6-39c414e87294 None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=5a58f9101f2d4785b96f0cc55be4330b) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:16.001249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c32c2-61c5-4cfd-aeb6-39c414e87294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:16.003532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c32c2-61c5-4cfd-aeb6-39c414e87294 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:16.012024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 262/541] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:15 2022] GET /identity/v3/role_assignments?user.id=5a58f9101f2d4785b96f0cc55be4330b&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:16.019349 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c32c2-61c5-4cfd-aeb6-39c414e87294 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:16.024556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:16.195382 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c32c2-61c5-4cfd-aeb6-39c414e87294 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:16.206748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683ee35-0133-4bf5-9633-95ae80772001 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:16.270788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:16.275734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9418d3-65ca-4c73-ba38-e49be6201022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:16.276367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9418d3-65ca-4c73-ba38-e49be6201022 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:16.276644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9418d3-65ca-4c73-ba38-e49be6201022 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:16.276912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9418d3-65ca-4c73-ba38-e49be6201022 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:16.282017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed519bc3-4a12-4561-b410-141f61bdbcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:16.289105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed519bc3-4a12-4561-b410-141f61bdbcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:16.289105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed519bc3-4a12-4561-b410-141f61bdbcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:16.289105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed519bc3-4a12-4561-b410-141f61bdbcf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:16.289105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed519bc3-4a12-4561-b410-141f61bdbcf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:16.289105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed519bc3-4a12-4561-b410-141f61bdbcf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:16.289105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed519bc3-4a12-4561-b410-141f61bdbcf8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:16.290881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9418d3-65ca-4c73-ba38-e49be6201022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-ad9418d3-65ca-4c73-ba38-e49be6201022 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:16.296526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:16.297415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:16.298509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 263/542] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:16.303717 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9418d3-65ca-4c73-ba38-e49be6201022 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:16.305055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 280/543] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:16.314041 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed519bc3-4a12-4561-b410-141f61bdbcf8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:16.374953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:16.415221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9418d3-65ca-4c73-ba38-e49be6201022 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:16.429448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed519bc3-4a12-4561-b410-141f61bdbcf8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:16.450691 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedff243-a0e9-4c7b-87de-ec5c3778384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:16.451292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedff243-a0e9-4c7b-87de-ec5c3778384d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:16.451569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedff243-a0e9-4c7b-87de-ec5c3778384d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:16.451837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedff243-a0e9-4c7b-87de-ec5c3778384d None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:16.452497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedff243-a0e9-4c7b-87de-ec5c3778384d None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:16.452795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedff243-a0e9-4c7b-87de-ec5c3778384d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:16.454328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedff243-a0e9-4c7b-87de-ec5c3778384d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:16.468219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 264/544] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:16.469610 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a35387-e42b-4e67-bbd1-222b2ec90e84 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:16.471637 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a35387-e42b-4e67-bbd1-222b2ec90e84 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:16.472602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7703b-3bbe-4eb0-a716-82e432f078bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:16.473430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a35387-e42b-4e67-bbd1-222b2ec90e84 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:16.473640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7703b-3bbe-4eb0-a716-82e432f078bf None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:16.474108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7703b-3bbe-4eb0-a716-82e432f078bf None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:16.474573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7703b-3bbe-4eb0-a716-82e432f078bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:16.475072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 265/545] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:16.475740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7703b-3bbe-4eb0-a716-82e432f078bf None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=88a7cdd610f74330a5da69c0de8472db) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:16.475853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a35387-e42b-4e67-bbd1-222b2ec90e84 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:16.476465 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7703b-3bbe-4eb0-a716-82e432f078bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:16.478178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7703b-3bbe-4eb0-a716-82e432f078bf None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:16.484484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 281/546] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:16 2022] GET /identity/v3/role_assignments?user.id=88a7cdd610f74330a5da69c0de8472db&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:16.486073 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f56b98-432f-49a6-846e-b607128f3a0e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:16.486583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f56b98-432f-49a6-846e-b607128f3a0e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:16.487058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f56b98-432f-49a6-846e-b607128f3a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:16.507264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-74f56b98-432f-49a6-846e-b607128f3a0e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:16.509388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-74f56b98-432f-49a6-846e-b607128f3a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:16.589934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:16.595749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 282/547] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:16.631600 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f56b98-432f-49a6-846e-b607128f3a0e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:16.660190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a35387-e42b-4e67-bbd1-222b2ec90e84 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:16.709176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:16.718475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7776a-8604-4752-a2c7-06440776f40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:16.719158 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7776a-8604-4752-a2c7-06440776f40c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:16.719438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7776a-8604-4752-a2c7-06440776f40c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:16.726598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7776a-8604-4752-a2c7-06440776f40c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:16.730516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7776a-8604-4752-a2c7-06440776f40c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:16.732037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7776a-8604-4752-a2c7-06440776f40c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-5ba7776a-8604-4752-a2c7-06440776f40c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:16.748085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:16.749115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:16.749989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:16.749989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:16.749989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:16.749989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 266/548] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:16.759544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7776a-8604-4752-a2c7-06440776f40c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:16.841455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f56b98-432f-49a6-846e-b607128f3a0e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:16.934324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44b307-5cf7-411d-8bdc-acf53a82ef5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:16.934973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44b307-5cf7-411d-8bdc-acf53a82ef5b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:16.935244 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44b307-5cf7-411d-8bdc-acf53a82ef5b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:16.935617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44b307-5cf7-411d-8bdc-acf53a82ef5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:16.959145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44b307-5cf7-411d-8bdc-acf53a82ef5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-dc44b307-5cf7-411d-8bdc-acf53a82ef5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:16.964380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:16.965463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:16.966304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 283/549] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:16.971568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7776a-8604-4752-a2c7-06440776f40c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:16.971707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44b307-5cf7-411d-8bdc-acf53a82ef5b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:17.048097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8ad24-108d-46f3-a13c-4fb74f741df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:17.048097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8ad24-108d-46f3-a13c-4fb74f741df9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:17.048097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8ad24-108d-46f3-a13c-4fb74f741df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:17.048097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8ad24-108d-46f3-a13c-4fb74f741df9 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:17.048825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8ad24-108d-46f3-a13c-4fb74f741df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:17.048825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8ad24-108d-46f3-a13c-4fb74f741df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:17.050186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8ad24-108d-46f3-a13c-4fb74f741df9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:17.068192 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 267/550] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:17.077548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8ad24-108d-46f3-a13c-4fb74f741df9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:17.089547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:17.123697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:17.201550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8ad24-108d-46f3-a13c-4fb74f741df9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:17.238417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc38ed9-dff4-4121-a838-ad26f723a7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:17.239857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc38ed9-dff4-4121-a838-ad26f723a7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:17.239857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc38ed9-dff4-4121-a838-ad26f723a7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:17.240090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc38ed9-dff4-4121-a838-ad26f723a7f3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:17.248750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc38ed9-dff4-4121-a838-ad26f723a7f3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:17.250902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc38ed9-dff4-4121-a838-ad26f723a7f3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-fbc38ed9-dff4-4121-a838-ad26f723a7f3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:17.269548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:17.270596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:17.271498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:17.271498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:17.271498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:17.271498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:17.271498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:17.271498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:17.271498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:17.271498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:17.271498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 18:30:17.271498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:17.271498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 268/551] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:17.275890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc38ed9-dff4-4121-a838-ad26f723a7f3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:17.311490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44b307-5cf7-411d-8bdc-acf53a82ef5b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:17.385979 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:17.391673 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc38ed9-dff4-4121-a838-ad26f723a7f3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:17.433204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e37cb-6dff-4961-aef1-63ee28461b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:17.433862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e37cb-6dff-4961-aef1-63ee28461b55 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:17.434136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e37cb-6dff-4961-aef1-63ee28461b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:17.434403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e37cb-6dff-4961-aef1-63ee28461b55 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:17.435135 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e37cb-6dff-4961-aef1-63ee28461b55 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:17.435436 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e37cb-6dff-4961-aef1-63ee28461b55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:17.436870 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e37cb-6dff-4961-aef1-63ee28461b55 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:17.456102 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 269/552] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Aug 9 18:30:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:17.457317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae9d7e-af48-493f-a1f2-19bca92fcc5a None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:17.462111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae9d7e-af48-493f-a1f2-19bca92fcc5a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:17.463423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae9d7e-af48-493f-a1f2-19bca92fcc5a None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:17.463919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 270/553] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:17.468066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae9d7e-af48-493f-a1f2-19bca92fcc5a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:17.480491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e494499-b186-4785-966f-fb1a1d5d8fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:17.483209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e494499-b186-4785-966f-fb1a1d5d8fae None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:17.483996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e494499-b186-4785-966f-fb1a1d5d8fae None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:17.484500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e494499-b186-4785-966f-fb1a1d5d8fae None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:17.486756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e494499-b186-4785-966f-fb1a1d5d8fae None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:17.487424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e494499-b186-4785-966f-fb1a1d5d8fae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:17.494998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e494499-b186-4785-966f-fb1a1d5d8fae None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:17.531476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 284/554] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 560 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:17.537092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa97ef8-1945-4aba-91e1-c78274a3a327 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:17.538481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa97ef8-1945-4aba-91e1-c78274a3a327 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:17.539805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa97ef8-1945-4aba-91e1-c78274a3a327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:17.618919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2fa97ef8-1945-4aba-91e1-c78274a3a327 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:17.623206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2fa97ef8-1945-4aba-91e1-c78274a3a327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:17.694227 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae9d7e-af48-493f-a1f2-19bca92fcc5a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:17.744033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 285/555] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:17.745223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa97ef8-1945-4aba-91e1-c78274a3a327 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:17.816338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:17.832693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b56de1-62ce-4e39-9f97-da27ed50f835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:17.833361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b56de1-62ce-4e39-9f97-da27ed50f835 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:17.839801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b56de1-62ce-4e39-9f97-da27ed50f835 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:17.839801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b56de1-62ce-4e39-9f97-da27ed50f835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:17.903203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa97ef8-1945-4aba-91e1-c78274a3a327 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:17.919282 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:17.951643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93026234-9d3e-4b71-abb0-06838ad51196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:17.952277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93026234-9d3e-4b71-abb0-06838ad51196 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:17.952556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93026234-9d3e-4b71-abb0-06838ad51196 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:17.952825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93026234-9d3e-4b71-abb0-06838ad51196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:17.974900 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93026234-9d3e-4b71-abb0-06838ad51196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:17.976374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93026234-9d3e-4b71-abb0-06838ad51196 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-93026234-9d3e-4b71-abb0-06838ad51196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:18.003135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:18.004067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:18.005021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:18.005021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:18.005021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:18.005021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 286/556] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:18.005829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93026234-9d3e-4b71-abb0-06838ad51196 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:18.016157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b56de1-62ce-4e39-9f97-da27ed50f835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:18.017826 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b56de1-62ce-4e39-9f97-da27ed50f835 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:18.199373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:18.224291 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93026234-9d3e-4b71-abb0-06838ad51196 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:18.228946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 271/557] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 764 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:18.230307 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b56de1-62ce-4e39-9f97-da27ed50f835 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:18.305921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54208a73-c438-4993-a602-92b9a97df4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:18.306797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54208a73-c438-4993-a602-92b9a97df4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:18.307412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54208a73-c438-4993-a602-92b9a97df4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:18.307975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54208a73-c438-4993-a602-92b9a97df4d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:18.330431 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54208a73-c438-4993-a602-92b9a97df4d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-54208a73-c438-4993-a602-92b9a97df4d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:18.335497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:18.337007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:18.337827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:18.337827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:18.337827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:18.337827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:18.337827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:18.337827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:18.337827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:18.337827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:18.337827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:18.337827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:18.337827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:18.337827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:18.344820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 287/558] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:18.346117 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54208a73-c438-4993-a602-92b9a97df4d6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:18.360636 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:18.413351 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b56de1-62ce-4e39-9f97-da27ed50f835 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:18.450910 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7a211-4dd0-42d7-945c-4bade1b930b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:18.451581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7a211-4dd0-42d7-945c-4bade1b930b9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:18.453033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7a211-4dd0-42d7-945c-4bade1b930b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:18.453033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7a211-4dd0-42d7-945c-4bade1b930b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:18.453033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7a211-4dd0-42d7-945c-4bade1b930b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:18.453248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7a211-4dd0-42d7-945c-4bade1b930b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:18.454775 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7a211-4dd0-42d7-945c-4bade1b930b9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:18.473363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 272/559] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:18.475119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7a211-4dd0-42d7-945c-4bade1b930b9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:18.479337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54208a73-c438-4993-a602-92b9a97df4d6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:18.514669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5b714-7eb1-4237-b7e3-1a7019e10584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:18.515670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5b714-7eb1-4237-b7e3-1a7019e10584 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:18.516157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5b714-7eb1-4237-b7e3-1a7019e10584 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:18.516683 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5b714-7eb1-4237-b7e3-1a7019e10584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:18.527948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5b714-7eb1-4237-b7e3-1a7019e10584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:18.529793 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5b714-7eb1-4237-b7e3-1a7019e10584 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:18.540516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-4eb5b714-7eb1-4237-b7e3-1a7019e10584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:18.547057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:18.548078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:18.548966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:18.548966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:18.548966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:18.551102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 288/560] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:18.552510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdafd57-b402-491b-a9ef-f066fe462c55 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:18.552984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdafd57-b402-491b-a9ef-f066fe462c55 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:18.556294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdafd57-b402-491b-a9ef-f066fe462c55 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:18.556294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 289/561] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:18.556294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdafd57-b402-491b-a9ef-f066fe462c55 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:18.599162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7a211-4dd0-42d7-945c-4bade1b930b9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:18.654084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:18.673823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdafd57-b402-491b-a9ef-f066fe462c55 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:18.676088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b958fc3-5d50-4646-9d64-616cbcfa211c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:18.676762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b958fc3-5d50-4646-9d64-616cbcfa211c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:18.677045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b958fc3-5d50-4646-9d64-616cbcfa211c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:18.677400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b958fc3-5d50-4646-9d64-616cbcfa211c None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:18.678097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b958fc3-5d50-4646-9d64-616cbcfa211c None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:18.678399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b958fc3-5d50-4646-9d64-616cbcfa211c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:18.680230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b958fc3-5d50-4646-9d64-616cbcfa211c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:18.694193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 273/562] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:18.695095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b958fc3-5d50-4646-9d64-616cbcfa211c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:18.734020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec94fb-9a50-4c5a-81be-2e50790341bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:18.734589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec94fb-9a50-4c5a-81be-2e50790341bf None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:18.734869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec94fb-9a50-4c5a-81be-2e50790341bf None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:18.735150 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec94fb-9a50-4c5a-81be-2e50790341bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:18.744224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec94fb-9a50-4c5a-81be-2e50790341bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:18.746798 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec94fb-9a50-4c5a-81be-2e50790341bf None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-06ec94fb-9a50-4c5a-81be-2e50790341bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:18.761227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:18.762400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:18.763359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:18.763359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:18.763359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:18.763359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:18.763359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:18.763359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:18.763359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:18.763359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:18.763359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:30:18.763359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:18.763359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 290/563] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 9 18:30:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:18.763949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775cbcf-a376-421e-953f-a6cec9207523 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:18.764251 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775cbcf-a376-421e-953f-a6cec9207523 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:18.764450 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775cbcf-a376-421e-953f-a6cec9207523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:18.783644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c775cbcf-a376-421e-953f-a6cec9207523 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:18.784839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c775cbcf-a376-421e-953f-a6cec9207523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:18.841570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b958fc3-5d50-4646-9d64-616cbcfa211c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:18.848565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:18.859812 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 291/564] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:18.860648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775cbcf-a376-421e-953f-a6cec9207523 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:18.886185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc182f4-3c31-4a1c-9352-d44ac09123ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:18.886791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc182f4-3c31-4a1c-9352-d44ac09123ab None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:18.887212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc182f4-3c31-4a1c-9352-d44ac09123ab None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:18.887489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc182f4-3c31-4a1c-9352-d44ac09123ab None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:18.888146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc182f4-3c31-4a1c-9352-d44ac09123ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:18.888452 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc182f4-3c31-4a1c-9352-d44ac09123ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:18.889860 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc182f4-3c31-4a1c-9352-d44ac09123ab None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:18.908763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 274/565] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:18.910482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4645ca3-b19c-4f5f-80f6-4d46735dd4a7 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:18.910772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4645ca3-b19c-4f5f-80f6-4d46735dd4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:18.911154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4645ca3-b19c-4f5f-80f6-4d46735dd4a7 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:18.912363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 275/566] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:18.913270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4645ca3-b19c-4f5f-80f6-4d46735dd4a7 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:18.962946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:18.982780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775cbcf-a376-421e-953f-a6cec9207523 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:19.021453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e1052-5aef-4965-ab17-cb7a4b4dfd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:19.022098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e1052-5aef-4965-ab17-cb7a4b4dfd48 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:19.022340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e1052-5aef-4965-ab17-cb7a4b4dfd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:19.022606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e1052-5aef-4965-ab17-cb7a4b4dfd48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:19.030888 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4645ca3-b19c-4f5f-80f6-4d46735dd4a7 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:19.033135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e1052-5aef-4965-ab17-cb7a4b4dfd48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:19.034641 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e1052-5aef-4965-ab17-cb7a4b4dfd48 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-198e1052-5aef-4965-ab17-cb7a4b4dfd48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:19.050816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:19.051800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:19.052686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:19.052686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:19.052686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:19.052686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 292/567] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:19.052686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e1052-5aef-4965-ab17-cb7a4b4dfd48 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:19.075260 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05ebf8-0559-4f9e-95ab-31e3e28f380b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:19.075862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05ebf8-0559-4f9e-95ab-31e3e28f380b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:19.076190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05ebf8-0559-4f9e-95ab-31e3e28f380b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:19.076468 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05ebf8-0559-4f9e-95ab-31e3e28f380b None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:19.077199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05ebf8-0559-4f9e-95ab-31e3e28f380b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:19.077509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05ebf8-0559-4f9e-95ab-31e3e28f380b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:19.078961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05ebf8-0559-4f9e-95ab-31e3e28f380b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:19.097331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 276/568] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 9 18:30:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:19.098190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05ebf8-0559-4f9e-95ab-31e3e28f380b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:19.122727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:19.181376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e1052-5aef-4965-ab17-cb7a4b4dfd48 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:19.192069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:19.198411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05ebf8-0559-4f9e-95ab-31e3e28f380b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:19.215458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff3d65-e70b-4ad2-b48b-9d061217c217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:19.216080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff3d65-e70b-4ad2-b48b-9d061217c217 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:19.216361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff3d65-e70b-4ad2-b48b-9d061217c217 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:19.216632 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff3d65-e70b-4ad2-b48b-9d061217c217 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:19.229232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff3d65-e70b-4ad2-b48b-9d061217c217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:19.234037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3438b-13fe-46ef-9754-da9383b4a15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-c6ff3d65-e70b-4ad2-b48b-9d061217c217 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:19.234472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:19.235393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:19.236395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:19.236395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:19.236395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:19.236395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:19.236395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:19.236395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:19.236395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:19.236395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:19.236395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:19.236395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:19.236395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:19.236395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:19.237230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3438b-13fe-46ef-9754-da9383b4a15e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:19.237230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3438b-13fe-46ef-9754-da9383b4a15e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:19.237230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3438b-13fe-46ef-9754-da9383b4a15e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:19.237406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 293/569] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:19.237406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2e8c3-fdb6-4c32-8be9-45a82fba43a0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:19.237406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2e8c3-fdb6-4c32-8be9-45a82fba43a0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:19.237406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2e8c3-fdb6-4c32-8be9-45a82fba43a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:19.246971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3438b-13fe-46ef-9754-da9383b4a15e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:19.248402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3438b-13fe-46ef-9754-da9383b4a15e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:19.255240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-41c2e8c3-fdb6-4c32-8be9-45a82fba43a0 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:19.256425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-41c2e8c3-fdb6-4c32-8be9-45a82fba43a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-ecd3438b-13fe-46ef-9754-da9383b4a15e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:19.259476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:19.260470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:19.262236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:19.262236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:19.262236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:19.262236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:19.262236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:19.262236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:19.262236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:19.262236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:19.262236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 277/570] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:19.262236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3438b-13fe-46ef-9754-da9383b4a15e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:19.331061 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 294/571] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:19.332202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2e8c3-fdb6-4c32-8be9-45a82fba43a0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:19.365288 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:19.366314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3438b-13fe-46ef-9754-da9383b4a15e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:19.400544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fce96-5b59-42fc-b679-f4f20a28d0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:19.401138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fce96-5b59-42fc-b679-f4f20a28d0ad None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:19.401421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fce96-5b59-42fc-b679-f4f20a28d0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:19.401725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fce96-5b59-42fc-b679-f4f20a28d0ad None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:19.402368 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318fce96-5b59-42fc-b679-f4f20a28d0ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:19.402674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318fce96-5b59-42fc-b679-f4f20a28d0ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:19.404135 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318fce96-5b59-42fc-b679-f4f20a28d0ad None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:19.415651 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 278/572] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:19.416513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fce96-5b59-42fc-b679-f4f20a28d0ad None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:19.434131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2e8c3-fdb6-4c32-8be9-45a82fba43a0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:19.434378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:19.470043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc667e-c73f-409b-8e16-dd77412877f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:19.470671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc667e-c73f-409b-8e16-dd77412877f7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:19.470949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc667e-c73f-409b-8e16-dd77412877f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:19.471218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc667e-c73f-409b-8e16-dd77412877f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:19.482067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc667e-c73f-409b-8e16-dd77412877f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:19.483518 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc667e-c73f-409b-8e16-dd77412877f7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-46bc667e-c73f-409b-8e16-dd77412877f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:19.498274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:19.499350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:19.500512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:19.500512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:19.500512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:19.500512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 295/573] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:19.500512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc667e-c73f-409b-8e16-dd77412877f7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:19.519898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fce96-5b59-42fc-b679-f4f20a28d0ad None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:19.555306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15539b9b-981f-4aa2-af57-18656a254c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:19.555911 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15539b9b-981f-4aa2-af57-18656a254c86 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:19.556193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15539b9b-981f-4aa2-af57-18656a254c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:19.556466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15539b9b-981f-4aa2-af57-18656a254c86 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:19.557129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15539b9b-981f-4aa2-af57-18656a254c86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:19.557434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15539b9b-981f-4aa2-af57-18656a254c86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:19.558833 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15539b9b-981f-4aa2-af57-18656a254c86 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:19.566880 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 279/574] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:19.567691 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15539b9b-981f-4aa2-af57-18656a254c86 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:19.592480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:19.608883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc667e-c73f-409b-8e16-dd77412877f7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:19.645832 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e83eb-8739-48e7-8935-538c70dffc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:19.646444 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e83eb-8739-48e7-8935-538c70dffc3e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:19.646720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e83eb-8739-48e7-8935-538c70dffc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:19.646992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e83eb-8739-48e7-8935-538c70dffc3e None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:19.647688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e83eb-8739-48e7-8935-538c70dffc3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:19.647992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e83eb-8739-48e7-8935-538c70dffc3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:19.649941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e83eb-8739-48e7-8935-538c70dffc3e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:19.652621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:19.662244 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 296/575] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:19.667609 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e83eb-8739-48e7-8935-538c70dffc3e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:19.675767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15539b9b-981f-4aa2-af57-18656a254c86 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:19.714774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf6199-db0d-43b7-9b03-ea7eed1bd7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:19.715401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf6199-db0d-43b7-9b03-ea7eed1bd7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:19.715681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf6199-db0d-43b7-9b03-ea7eed1bd7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:19.715959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf6199-db0d-43b7-9b03-ea7eed1bd7a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:19.727492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf6199-db0d-43b7-9b03-ea7eed1bd7a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-44cf6199-db0d-43b7-9b03-ea7eed1bd7a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:19.733542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:19.734557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 280/576] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:19.735405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf6199-db0d-43b7-9b03-ea7eed1bd7a4 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:19.785241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e83eb-8739-48e7-8935-538c70dffc3e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:19.822633 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0425c4a-fb0a-4dac-8050-f39fd095a3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:19.823268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0425c4a-fb0a-4dac-8050-f39fd095a3af None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:19.823549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0425c4a-fb0a-4dac-8050-f39fd095a3af None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:19.823823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0425c4a-fb0a-4dac-8050-f39fd095a3af None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:19.831655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0425c4a-fb0a-4dac-8050-f39fd095a3af None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:19.833105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0425c4a-fb0a-4dac-8050-f39fd095a3af None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f0425c4a-fb0a-4dac-8050-f39fd095a3af None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:19.845415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:19.846421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:19.847360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:19.847360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:19.847360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:19.847360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:19.847360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:19.847360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:19.847360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:19.847360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:19.847360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 18:30:19.847360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:19.847896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf6199-db0d-43b7-9b03-ea7eed1bd7a4 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:19.847978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 297/577] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:19.849160 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0425c4a-fb0a-4dac-8050-f39fd095a3af None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:19.856087 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:19.886401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada44ff7-823d-4005-b90e-0986842436d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:19.887181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada44ff7-823d-4005-b90e-0986842436d6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:19.887454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada44ff7-823d-4005-b90e-0986842436d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:19.887725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada44ff7-823d-4005-b90e-0986842436d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:19.888378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada44ff7-823d-4005-b90e-0986842436d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:19.888681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada44ff7-823d-4005-b90e-0986842436d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:19.891054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada44ff7-823d-4005-b90e-0986842436d6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:19.900227 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:19.904076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 281/578] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:19.904954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada44ff7-823d-4005-b90e-0986842436d6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:19.978701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0425c4a-fb0a-4dac-8050-f39fd095a3af None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:20.014135 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada44ff7-823d-4005-b90e-0986842436d6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:20.015276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84e89e-791a-467f-8fc2-fb4d85f0e648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:20.015531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84e89e-791a-467f-8fc2-fb4d85f0e648 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:20.015819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84e89e-791a-467f-8fc2-fb4d85f0e648 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:20.016093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84e89e-791a-467f-8fc2-fb4d85f0e648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:20.034969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84e89e-791a-467f-8fc2-fb4d85f0e648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:20.036511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84e89e-791a-467f-8fc2-fb4d85f0e648 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-0f84e89e-791a-467f-8fc2-fb4d85f0e648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:20.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:20.052899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:20.053793 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:20.053793 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:20.053793 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:20.053793 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 298/579] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:20.054012 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84e89e-791a-467f-8fc2-fb4d85f0e648 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:20.056417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b40eb9-4814-494b-abb7-7447439ebd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:20.057027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b40eb9-4814-494b-abb7-7447439ebd9b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:20.057307 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b40eb9-4814-494b-abb7-7447439ebd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:20.058785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b40eb9-4814-494b-abb7-7447439ebd9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:20.058785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b40eb9-4814-494b-abb7-7447439ebd9b None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=2ff735ecfcdf49089b5cfeafc56048cf) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:20.059112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b40eb9-4814-494b-abb7-7447439ebd9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:20.060446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b40eb9-4814-494b-abb7-7447439ebd9b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:20.066931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 282/580] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:19 2022] GET /identity/v3/role_assignments?user.id=2ff735ecfcdf49089b5cfeafc56048cf&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:20.067705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b40eb9-4814-494b-abb7-7447439ebd9b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:20.176868 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:20.181210 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:20.232208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b40eb9-4814-494b-abb7-7447439ebd9b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:20.239326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84e89e-791a-467f-8fc2-fb4d85f0e648 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:20.274380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8739cba-8279-40df-b4dd-3caaf0cbbfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:20.274998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8739cba-8279-40df-b4dd-3caaf0cbbfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:20.275279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8739cba-8279-40df-b4dd-3caaf0cbbfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:20.275547 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8739cba-8279-40df-b4dd-3caaf0cbbfc5 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:20.276279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8739cba-8279-40df-b4dd-3caaf0cbbfc5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:20.276681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8739cba-8279-40df-b4dd-3caaf0cbbfc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:20.278059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8739cba-8279-40df-b4dd-3caaf0cbbfc5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:20.294558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c1339-0966-44cf-b7ef-507ee978a9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:20.295658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c1339-0966-44cf-b7ef-507ee978a9be None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:20.295658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c1339-0966-44cf-b7ef-507ee978a9be None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:20.295877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c1339-0966-44cf-b7ef-507ee978a9be None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:20.296773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c1339-0966-44cf-b7ef-507ee978a9be None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:20.296773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c1339-0966-44cf-b7ef-507ee978a9be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:20.300905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c1339-0966-44cf-b7ef-507ee978a9be None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:20.301049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 283/581] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Aug 9 18:30:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:20.301049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8739cba-8279-40df-b4dd-3caaf0cbbfc5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:20.313571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 299/582] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:20.314258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c1339-0966-44cf-b7ef-507ee978a9be None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:20.439562 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8739cba-8279-40df-b4dd-3caaf0cbbfc5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:20.440115 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c1339-0966-44cf-b7ef-507ee978a9be None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:20.459376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:20.462939 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:20.482648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22853f99-da99-4229-b207-2fe2a4d7e93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:20.482897 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4509a2-77bb-4792-991b-0aa9a8e4c4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:20.483294 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22853f99-da99-4229-b207-2fe2a4d7e93e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:20.483395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4509a2-77bb-4792-991b-0aa9a8e4c4aa None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:20.483576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22853f99-da99-4229-b207-2fe2a4d7e93e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:20.483681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4509a2-77bb-4792-991b-0aa9a8e4c4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:20.483850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22853f99-da99-4229-b207-2fe2a4d7e93e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:20.483961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4509a2-77bb-4792-991b-0aa9a8e4c4aa None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:20.484551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4509a2-77bb-4792-991b-0aa9a8e4c4aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:20.484812 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4509a2-77bb-4792-991b-0aa9a8e4c4aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:20.486430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4509a2-77bb-4792-991b-0aa9a8e4c4aa None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:20.492010 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22853f99-da99-4229-b207-2fe2a4d7e93e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:20.493479 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22853f99-da99-4229-b207-2fe2a4d7e93e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:20.503715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 300/583] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:20.504650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4509a2-77bb-4792-991b-0aa9a8e4c4aa None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-22853f99-da99-4229-b207-2fe2a4d7e93e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:20.510246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:20.511532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:20.513040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:20.513040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:20.513040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:20.513040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:20.513040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:20.513040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:20.513040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:20.513040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:20.516522 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 284/584] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:20.518608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22853f99-da99-4229-b207-2fe2a4d7e93e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:20.631547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4509a2-77bb-4792-991b-0aa9a8e4c4aa None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:20.640146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22853f99-da99-4229-b207-2fe2a4d7e93e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:20.672703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b1850-eb43-470f-8b8f-093cee554b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:20.673338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b1850-eb43-470f-8b8f-093cee554b4d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:20.673654 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b1850-eb43-470f-8b8f-093cee554b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:20.673931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b1850-eb43-470f-8b8f-093cee554b4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:20.685766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b1850-eb43-470f-8b8f-093cee554b4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:20.687519 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b1850-eb43-470f-8b8f-093cee554b4d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:20.693063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20810e93-7f88-4c1e-866a-051d6ca5b72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:20.693662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20810e93-7f88-4c1e-866a-051d6ca5b72f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:20.693946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20810e93-7f88-4c1e-866a-051d6ca5b72f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:20.694221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20810e93-7f88-4c1e-866a-051d6ca5b72f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-019b1850-eb43-470f-8b8f-093cee554b4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:20.705239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:20.706230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:20.707093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:20.707093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:20.707093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:20.707259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20810e93-7f88-4c1e-866a-051d6ca5b72f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:20.707860 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20810e93-7f88-4c1e-866a-051d6ca5b72f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:20.709139 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 301/585] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:20.710189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b1850-eb43-470f-8b8f-093cee554b4d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-20810e93-7f88-4c1e-866a-051d6ca5b72f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:20.729431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:20.730523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:20.731376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:20.731376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:20.731376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:20.734387 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 285/586] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:20.735710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20810e93-7f88-4c1e-866a-051d6ca5b72f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:20.750365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:20.778901 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:20.841175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b1850-eb43-470f-8b8f-093cee554b4d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:20.859129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20810e93-7f88-4c1e-866a-051d6ca5b72f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:20.882902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc246b32-d90c-4d83-a32b-8525f1ec51b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:20.883590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc246b32-d90c-4d83-a32b-8525f1ec51b4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:20.883816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc246b32-d90c-4d83-a32b-8525f1ec51b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:20.884082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc246b32-d90c-4d83-a32b-8525f1ec51b4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:20.893724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc246b32-d90c-4d83-a32b-8525f1ec51b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:20.895185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc246b32-d90c-4d83-a32b-8525f1ec51b4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:20.899006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58783878-50e0-43a8-b719-e6f583099837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:20.899621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58783878-50e0-43a8-b719-e6f583099837 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:20.899893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58783878-50e0-43a8-b719-e6f583099837 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:20.900163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58783878-50e0-43a8-b719-e6f583099837 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:20.900887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58783878-50e0-43a8-b719-e6f583099837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:20.901219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58783878-50e0-43a8-b719-e6f583099837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:20.902687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58783878-50e0-43a8-b719-e6f583099837 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-dc246b32-d90c-4d83-a32b-8525f1ec51b4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:20.910200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:20.921132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:20.923568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:20.923568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:20.923568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:20.923568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:20.923568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:20.923568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:20.923568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:20.923568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:20.923568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 18:30:20.923568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:20.923568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 302/587] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:20.923568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc246b32-d90c-4d83-a32b-8525f1ec51b4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:20.928336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 286/588] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:20.930133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58783878-50e0-43a8-b719-e6f583099837 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:21.053429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:21.053640 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:21.053818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc246b32-d90c-4d83-a32b-8525f1ec51b4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:21.053897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58783878-50e0-43a8-b719-e6f583099837 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:21.091229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbefe4-19ed-4b83-9c77-0e3be09b0487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:21.091836 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbefe4-19ed-4b83-9c77-0e3be09b0487 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:21.092160 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbefe4-19ed-4b83-9c77-0e3be09b0487 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:21.092435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbefe4-19ed-4b83-9c77-0e3be09b0487 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:21.093148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbefe4-19ed-4b83-9c77-0e3be09b0487 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:21.093450 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbefe4-19ed-4b83-9c77-0e3be09b0487 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:21.094916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbefe4-19ed-4b83-9c77-0e3be09b0487 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:21.098916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49951ad7-1c65-487c-ac2d-508681ed52d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:21.099743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49951ad7-1c65-487c-ac2d-508681ed52d4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:21.099963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49951ad7-1c65-487c-ac2d-508681ed52d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:21.101034 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49951ad7-1c65-487c-ac2d-508681ed52d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:21.101034 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49951ad7-1c65-487c-ac2d-508681ed52d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:21.101336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49951ad7-1c65-487c-ac2d-508681ed52d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:21.102593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49951ad7-1c65-487c-ac2d-508681ed52d4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:21.115110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 303/589] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:21.115110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbefe4-19ed-4b83-9c77-0e3be09b0487 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:21.121240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 287/590] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:21.122852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49951ad7-1c65-487c-ac2d-508681ed52d4 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:21.238996 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49951ad7-1c65-487c-ac2d-508681ed52d4 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:21.240830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbefe4-19ed-4b83-9c77-0e3be09b0487 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:21.297618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd44b51-9a17-4d82-a202-65bdb16b3f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:21.298692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd44b51-9a17-4d82-a202-65bdb16b3f05 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:21.299132 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f622dd-0cad-4d2e-863e-0cff9eaaf979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:21.299902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd44b51-9a17-4d82-a202-65bdb16b3f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:21.300109 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f622dd-0cad-4d2e-863e-0cff9eaaf979 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:21.300534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f622dd-0cad-4d2e-863e-0cff9eaaf979 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:21.300868 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd44b51-9a17-4d82-a202-65bdb16b3f05 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:21.301013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f622dd-0cad-4d2e-863e-0cff9eaaf979 None admin] PATH_INFO: `/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/e2ecd7d1cce644b0982cc23740537933/roles/106b56dfde9d4c4f9f32ca4bec0b3e87` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:21.302101 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd44b51-9a17-4d82-a202-65bdb16b3f05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:21.302683 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd44b51-9a17-4d82-a202-65bdb16b3f05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:21.305468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd44b51-9a17-4d82-a202-65bdb16b3f05 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:21.324589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f622dd-0cad-4d2e-863e-0cff9eaaf979 None admin] RBAC: Authorizing `identity:create_grant(project_id=abcccf694e7b4f66a7372e0ede416caf, user_id=e2ecd7d1cce644b0982cc23740537933, role_id=106b56dfde9d4c4f9f32ca4bec0b3e87)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:21.326909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f622dd-0cad-4d2e-863e-0cff9eaaf979 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:21.332447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 304/591] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Aug 9 18:30:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:21.332447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd44b51-9a17-4d82-a202-65bdb16b3f05 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:21.344302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:21.402174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 288/592] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:30:21 2022] PUT /identity/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/e2ecd7d1cce644b0982cc23740537933/roles/106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:30:21.403047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f622dd-0cad-4d2e-863e-0cff9eaaf979 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:21.405942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:21.492995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd44b51-9a17-4d82-a202-65bdb16b3f05 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:21.546207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ebec8-4c11-49f4-8fd2-afd58d4f2723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:21.546862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ebec8-4c11-49f4-8fd2-afd58d4f2723 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:21.547145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ebec8-4c11-49f4-8fd2-afd58d4f2723 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:21.547416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ebec8-4c11-49f4-8fd2-afd58d4f2723 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:21.557142 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ebec8-4c11-49f4-8fd2-afd58d4f2723 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:21.558716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ebec8-4c11-49f4-8fd2-afd58d4f2723 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:21.563164 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f622dd-0cad-4d2e-863e-0cff9eaaf979 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-6c3ebec8-4c11-49f4-8fd2-afd58d4f2723 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:21.580296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:21.581442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:21.582560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:21.582560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:21.582560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:21.582560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:21.582560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:21.582560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:21.582560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:21.582560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:21.582560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 305/593] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:21.583140 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ebec8-4c11-49f4-8fd2-afd58d4f2723 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:21.623411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca1d4d-6a4a-41bb-aebe-fc909f116ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:21.624350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca1d4d-6a4a-41bb-aebe-fc909f116ade None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:21.624727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca1d4d-6a4a-41bb-aebe-fc909f116ade None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:21.625191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca1d4d-6a4a-41bb-aebe-fc909f116ade None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:21.636754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca1d4d-6a4a-41bb-aebe-fc909f116ade None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:21.638509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca1d4d-6a4a-41bb-aebe-fc909f116ade None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-6dca1d4d-6a4a-41bb-aebe-fc909f116ade None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:21.651000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:21.652047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:21.652977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:21.652977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:21.652977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:21.652977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:21.652977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:21.652977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:21.652977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:21.652977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:21.652977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 18:30:21.652977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:21.655549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 289/594] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:21.656511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca1d4d-6a4a-41bb-aebe-fc909f116ade None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:21.674719 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:21.720914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ebec8-4c11-49f4-8fd2-afd58d4f2723 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:21.747077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:21.772029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dbdc6-d07a-471e-9c3b-0df055428a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:21.772741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dbdc6-d07a-471e-9c3b-0df055428a26 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:21.773021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dbdc6-d07a-471e-9c3b-0df055428a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:21.773320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dbdc6-d07a-471e-9c3b-0df055428a26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:21.789834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213dbdc6-d07a-471e-9c3b-0df055428a26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:21.792017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213dbdc6-d07a-471e-9c3b-0df055428a26 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:21.800240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca1d4d-6a4a-41bb-aebe-fc909f116ade None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-213dbdc6-d07a-471e-9c3b-0df055428a26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:21.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:21.812776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:21.813742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:21.813742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:21.813742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:21.813742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 306/595] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:21.817050 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dbdc6-d07a-471e-9c3b-0df055428a26 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:21.850445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6154e9-c69c-47af-91a8-30bb950d6137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:21.852148 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6154e9-c69c-47af-91a8-30bb950d6137 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:21.852977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6154e9-c69c-47af-91a8-30bb950d6137 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:21.853620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6154e9-c69c-47af-91a8-30bb950d6137 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:21.854628 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6154e9-c69c-47af-91a8-30bb950d6137 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:21.855275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6154e9-c69c-47af-91a8-30bb950d6137 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:21.857513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6154e9-c69c-47af-91a8-30bb950d6137 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:21.881449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 290/596] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:21.882648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6154e9-c69c-47af-91a8-30bb950d6137 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:21.934511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dbdc6-d07a-471e-9c3b-0df055428a26 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:21.957214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:21.972131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62371868-d490-4fe6-8cbd-78c72aec32d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:21.972757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62371868-d490-4fe6-8cbd-78c72aec32d8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:21.973099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62371868-d490-4fe6-8cbd-78c72aec32d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:21.973854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62371868-d490-4fe6-8cbd-78c72aec32d8 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:21.974416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62371868-d490-4fe6-8cbd-78c72aec32d8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:21.974898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62371868-d490-4fe6-8cbd-78c72aec32d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:21.977072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62371868-d490-4fe6-8cbd-78c72aec32d8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:21.997069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 307/597] 10.222.0.57 () {56 vars in 1139 bytes} [Tue Aug 9 18:30:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:21.997946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62371868-d490-4fe6-8cbd-78c72aec32d8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:22.000065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6154e9-c69c-47af-91a8-30bb950d6137 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:22.031945 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:22.043395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df410ec-21b2-4ee3-a003-81d2fa9b5a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:22.044208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df410ec-21b2-4ee3-a003-81d2fa9b5a58 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:22.044629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df410ec-21b2-4ee3-a003-81d2fa9b5a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:22.048645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df410ec-21b2-4ee3-a003-81d2fa9b5a58 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:22.048645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df410ec-21b2-4ee3-a003-81d2fa9b5a58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:22.048645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df410ec-21b2-4ee3-a003-81d2fa9b5a58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:22.048645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df410ec-21b2-4ee3-a003-81d2fa9b5a58 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:22.062959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 291/598] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:22.064385 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df410ec-21b2-4ee3-a003-81d2fa9b5a58 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:22.112590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62371868-d490-4fe6-8cbd-78c72aec32d8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:22.151194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d9a8e-6da4-4150-b094-2e15ea103898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:22.151940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d9a8e-6da4-4150-b094-2e15ea103898 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:22.152218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d9a8e-6da4-4150-b094-2e15ea103898 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:22.152493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d9a8e-6da4-4150-b094-2e15ea103898 None admin] PATH_INFO: `/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/77300e25b9004a598f8f7f12f3cf4866/roles/106b56dfde9d4c4f9f32ca4bec0b3e87` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:22.168633 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d9a8e-6da4-4150-b094-2e15ea103898 None admin] RBAC: Authorizing `identity:create_grant(project_id=abcccf694e7b4f66a7372e0ede416caf, user_id=77300e25b9004a598f8f7f12f3cf4866, role_id=106b56dfde9d4c4f9f32ca4bec0b3e87)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:22.170481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d9a8e-6da4-4150-b094-2e15ea103898 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:22.188115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df410ec-21b2-4ee3-a003-81d2fa9b5a58 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:22.223221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fa220-86b6-4f27-94df-6091108c3683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:22.223848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fa220-86b6-4f27-94df-6091108c3683 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:22.224125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fa220-86b6-4f27-94df-6091108c3683 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:22.224395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fa220-86b6-4f27-94df-6091108c3683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:22.238370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fa220-86b6-4f27-94df-6091108c3683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:22.241116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 308/599] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:30:21 2022] PUT /identity/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/77300e25b9004a598f8f7f12f3cf4866/roles/106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:30:22.241116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d9a8e-6da4-4150-b094-2e15ea103898 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:22.250313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fa220-86b6-4f27-94df-6091108c3683 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-a55fa220-86b6-4f27-94df-6091108c3683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:22.272428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:22.273999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:22.275607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:22.275607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:22.275607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:22.275607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 292/600] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:22.276284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293ab6a-bde7-4915-b262-3012cce0777e None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:22.276431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293ab6a-bde7-4915-b262-3012cce0777e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:22.276696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293ab6a-bde7-4915-b262-3012cce0777e None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:22.282608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 293/601] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:22.286309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76bb57-c090-4e1b-aa27-c1ab1c62585e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:22.286883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76bb57-c090-4e1b-aa27-c1ab1c62585e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:22.287457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76bb57-c090-4e1b-aa27-c1ab1c62585e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:22.299886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:22.320288 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1b76bb57-c090-4e1b-aa27-c1ab1c62585e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:22.320288 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1b76bb57-c090-4e1b-aa27-c1ab1c62585e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:22.368089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:22.414458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d9a8e-6da4-4150-b094-2e15ea103898 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:22.432006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 294/602] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:22.432884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76bb57-c090-4e1b-aa27-c1ab1c62585e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:22.459640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aba37-7411-4727-8c94-f9f97a41a78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:22.460436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aba37-7411-4727-8c94-f9f97a41a78a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:22.461497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aba37-7411-4727-8c94-f9f97a41a78a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:22.461829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aba37-7411-4727-8c94-f9f97a41a78a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:22.472967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aba37-7411-4727-8c94-f9f97a41a78a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:22.474674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aba37-7411-4727-8c94-f9f97a41a78a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-503aba37-7411-4727-8c94-f9f97a41a78a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:22.491185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:22.492378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:22.493377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:22.493377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:22.493377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:22.493377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:22.493377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:22.493377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:22.493377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:22.493377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:22.493377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 309/603] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:22.498068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aba37-7411-4727-8c94-f9f97a41a78a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:22.578684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76bb57-c090-4e1b-aa27-c1ab1c62585e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:22.619509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301520e4-3bb5-4f20-930f-e8ca8f6dc8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:22.620325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301520e4-3bb5-4f20-930f-e8ca8f6dc8fe None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:22.620735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301520e4-3bb5-4f20-930f-e8ca8f6dc8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:22.621014 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301520e4-3bb5-4f20-930f-e8ca8f6dc8fe None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:22.621703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301520e4-3bb5-4f20-930f-e8ca8f6dc8fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:22.622004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301520e4-3bb5-4f20-930f-e8ca8f6dc8fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:22.623869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301520e4-3bb5-4f20-930f-e8ca8f6dc8fe None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:22.634484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:22.641490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 295/604] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:22.642861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301520e4-3bb5-4f20-930f-e8ca8f6dc8fe None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:22.648901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aba37-7411-4727-8c94-f9f97a41a78a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:22.689176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:22.708311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4cea8-5c56-4951-a241-c54a9a217ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:22.709275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4cea8-5c56-4951-a241-c54a9a217ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:22.709733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4cea8-5c56-4951-a241-c54a9a217ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:22.710368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4cea8-5c56-4951-a241-c54a9a217ad4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:22.728432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4cea8-5c56-4951-a241-c54a9a217ad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b2e4cea8-5c56-4951-a241-c54a9a217ad4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:22.735661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:22.736810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:22.737855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:22.737855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:22.737855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:22.737855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:22.737855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:22.737855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:22.737855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:22.737855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:22.737855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:22.737855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:22.737855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:22.737855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:22.741433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 310/605] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:22.742482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4cea8-5c56-4951-a241-c54a9a217ad4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:22.802439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301520e4-3bb5-4f20-930f-e8ca8f6dc8fe None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:22.841405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94024d-6893-4758-9d05-b49aae8ba6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:22.841862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94024d-6893-4758-9d05-b49aae8ba6ff None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:22.842233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94024d-6893-4758-9d05-b49aae8ba6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:22.842598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94024d-6893-4758-9d05-b49aae8ba6ff None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:22.843394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94024d-6893-4758-9d05-b49aae8ba6ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:22.843802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94024d-6893-4758-9d05-b49aae8ba6ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:22.845310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94024d-6893-4758-9d05-b49aae8ba6ff None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:22.862852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 296/606] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:22.863893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94024d-6893-4758-9d05-b49aae8ba6ff None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:22.872721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4cea8-5c56-4951-a241-c54a9a217ad4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:22.920883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e7030-f1da-4d4e-96aa-944658af894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:22.921707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e7030-f1da-4d4e-96aa-944658af894d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:22.922082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e7030-f1da-4d4e-96aa-944658af894d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:22.922448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e7030-f1da-4d4e-96aa-944658af894d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:22.931159 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e7030-f1da-4d4e-96aa-944658af894d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:22.932921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e7030-f1da-4d4e-96aa-944658af894d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-5c0e7030-f1da-4d4e-96aa-944658af894d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:22.945970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:22.947053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:22.947924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:22.947924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:22.947924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:22.947924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:22.947924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:22.947924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:22.947924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:22.947924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:22.953151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 311/607] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:22.954596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e7030-f1da-4d4e-96aa-944658af894d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:22.971480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:22.981298 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:23.001728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94024d-6893-4758-9d05-b49aae8ba6ff None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:23.046326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a28915-e336-4b6b-9ca3-b9999dc02a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:23.047568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a28915-e336-4b6b-9ca3-b9999dc02a2d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:23.048454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a28915-e336-4b6b-9ca3-b9999dc02a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:23.049092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a28915-e336-4b6b-9ca3-b9999dc02a2d None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:23.050219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a28915-e336-4b6b-9ca3-b9999dc02a2d None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:23.051564 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a28915-e336-4b6b-9ca3-b9999dc02a2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:23.053153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a28915-e336-4b6b-9ca3-b9999dc02a2d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:23.065700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 297/608] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:23.067407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a28915-e336-4b6b-9ca3-b9999dc02a2d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:23.097115 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e7030-f1da-4d4e-96aa-944658af894d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:23.150446 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c91e61-bc26-4088-9560-610e40254d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:23.151112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c91e61-bc26-4088-9560-610e40254d5b None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:23.151399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c91e61-bc26-4088-9560-610e40254d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:23.151828 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c91e61-bc26-4088-9560-610e40254d5b None admin] PATH_INFO: `/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/5a58f9101f2d4785b96f0cc55be4330b/roles/106b56dfde9d4c4f9f32ca4bec0b3e87` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:23.170919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c91e61-bc26-4088-9560-610e40254d5b None admin] RBAC: Authorizing `identity:create_grant(project_id=abcccf694e7b4f66a7372e0ede416caf, user_id=5a58f9101f2d4785b96f0cc55be4330b, role_id=106b56dfde9d4c4f9f32ca4bec0b3e87)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:23.172366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c91e61-bc26-4088-9560-610e40254d5b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:23.200789 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a28915-e336-4b6b-9ca3-b9999dc02a2d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:23.243573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fe186-4160-40fd-993a-ed0abfbfde53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:23.244291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fe186-4160-40fd-993a-ed0abfbfde53 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:23.244636 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fe186-4160-40fd-993a-ed0abfbfde53 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:23.244992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fe186-4160-40fd-993a-ed0abfbfde53 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:23.245786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fe186-4160-40fd-993a-ed0abfbfde53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:23.246156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fe186-4160-40fd-993a-ed0abfbfde53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:23.247760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fe186-4160-40fd-993a-ed0abfbfde53 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:23.270457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 312/609] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:30:22 2022] PUT /identity/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/5a58f9101f2d4785b96f0cc55be4330b/roles/106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:30:23.272102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c91e61-bc26-4088-9560-610e40254d5b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:23.274739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 298/610] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:23.275776 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fe186-4160-40fd-993a-ed0abfbfde53 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:23.285980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:23.361790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:23.405931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fe186-4160-40fd-993a-ed0abfbfde53 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:23.414554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c91e61-bc26-4088-9560-610e40254d5b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:23.467922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54aadb-36c5-4ba4-adb6-7c253fd88bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:23.469000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54aadb-36c5-4ba4-adb6-7c253fd88bdd None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:23.469484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54aadb-36c5-4ba4-adb6-7c253fd88bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:23.470005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54aadb-36c5-4ba4-adb6-7c253fd88bdd None admin] PATH_INFO: `/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/88a7cdd610f74330a5da69c0de8472db/roles/106b56dfde9d4c4f9f32ca4bec0b3e87` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:23.482128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382d7ce-97ef-4d4f-8d45-5407defb3601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:23.482997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382d7ce-97ef-4d4f-8d45-5407defb3601 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:23.483411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382d7ce-97ef-4d4f-8d45-5407defb3601 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:23.483697 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382d7ce-97ef-4d4f-8d45-5407defb3601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:23.489217 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54aadb-36c5-4ba4-adb6-7c253fd88bdd None admin] RBAC: Authorizing `identity:create_grant(project_id=abcccf694e7b4f66a7372e0ede416caf, user_id=88a7cdd610f74330a5da69c0de8472db, role_id=106b56dfde9d4c4f9f32ca4bec0b3e87)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:23.491057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54aadb-36c5-4ba4-adb6-7c253fd88bdd None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:23.507745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382d7ce-97ef-4d4f-8d45-5407defb3601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:23.509754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382d7ce-97ef-4d4f-8d45-5407defb3601 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-a382d7ce-97ef-4d4f-8d45-5407defb3601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:23.526414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:23.527443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:23.528314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:23.528314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:23.528314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:23.534757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 313/611] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:23.537320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382d7ce-97ef-4d4f-8d45-5407defb3601 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:23.576744 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 299/612] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:30:23 2022] PUT /identity/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/88a7cdd610f74330a5da69c0de8472db/roles/106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:30:23.599089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48793ead-17a2-4f51-a3bd-6885c7e85ae8 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:23.599754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48793ead-17a2-4f51-a3bd-6885c7e85ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:23.600231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48793ead-17a2-4f51-a3bd-6885c7e85ae8 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:23.603019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 300/613] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:23.608247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad979b-f1fb-4fca-973f-46b09aae19b2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:23.608791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad979b-f1fb-4fca-973f-46b09aae19b2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:23.609314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad979b-f1fb-4fca-973f-46b09aae19b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:23.649482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ffad979b-f1fb-4fca-973f-46b09aae19b2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:23.651223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ffad979b-f1fb-4fca-973f-46b09aae19b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:23.686669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382d7ce-97ef-4d4f-8d45-5407defb3601 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:23.700172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:23.735268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10d06a-0cc3-4613-b30f-f160da008cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:23.736178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d06a-0cc3-4613-b30f-f160da008cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:23.738050 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d06a-0cc3-4613-b30f-f160da008cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:23.738513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d06a-0cc3-4613-b30f-f160da008cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:23.739339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10d06a-0cc3-4613-b30f-f160da008cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:23.740813 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10d06a-0cc3-4613-b30f-f160da008cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:23.743781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10d06a-0cc3-4613-b30f-f160da008cb5 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:23.768742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 314/614] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:30:23.774546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10d06a-0cc3-4613-b30f-f160da008cb5 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:23.776890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:23.808584 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 301/615] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:23.838207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad979b-f1fb-4fca-973f-46b09aae19b2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:23.947545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10d06a-0cc3-4613-b30f-f160da008cb5 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:24.013086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad979b-f1fb-4fca-973f-46b09aae19b2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:24.018705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc585985-42bf-43c2-98ef-13f494569b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:24.019353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc585985-42bf-43c2-98ef-13f494569b62 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:24.019655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc585985-42bf-43c2-98ef-13f494569b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:24.019922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc585985-42bf-43c2-98ef-13f494569b62 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:24.046732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc585985-42bf-43c2-98ef-13f494569b62 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:24.048216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc585985-42bf-43c2-98ef-13f494569b62 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:24.052204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6eec3a-54a9-4d7f-9847-b9d473b04bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:24.052842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6eec3a-54a9-4d7f-9847-b9d473b04bbf None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:24.053115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6eec3a-54a9-4d7f-9847-b9d473b04bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:24.053380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6eec3a-54a9-4d7f-9847-b9d473b04bbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-dc585985-42bf-43c2-98ef-13f494569b62 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:24.072642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:24.073670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:24.074602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:24.074602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:24.074602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:24.074602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:24.074602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:24.074602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:24.074602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:24.074602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:24.074602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 18:30:24.074602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:24.076761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 315/616] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:24.077722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6eec3a-54a9-4d7f-9847-b9d473b04bbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:24.079428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc585985-42bf-43c2-98ef-13f494569b62 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-3d6eec3a-54a9-4d7f-9847-b9d473b04bbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:24.084426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:24.090215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:24.092229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:24.092229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:24.092229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:24.092229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:24.092229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:24.092229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:24.092229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:24.092229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:24.092229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:24.092229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:24.092229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:24.092229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:24.097876 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 302/617] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:30:24.097876 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6eec3a-54a9-4d7f-9847-b9d473b04bbf None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:24.170007 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:24.205372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:24.224657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6eec3a-54a9-4d7f-9847-b9d473b04bbf None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:24.287727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32697a6-b641-45b3-9957-41c0ea871942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:24.288343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32697a6-b641-45b3-9957-41c0ea871942 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:24.288623 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32697a6-b641-45b3-9957-41c0ea871942 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:24.288895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32697a6-b641-45b3-9957-41c0ea871942 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:24.293697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32697a6-b641-45b3-9957-41c0ea871942 None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:24.294043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32697a6-b641-45b3-9957-41c0ea871942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:24.298327 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32697a6-b641-45b3-9957-41c0ea871942 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:24.312593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 303/618] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:24.318797 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32697a6-b641-45b3-9957-41c0ea871942 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:24.351686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc585985-42bf-43c2-98ef-13f494569b62 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:24.434293 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9533c-454a-4d16-8023-e5e904c78ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:24.434902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9533c-454a-4d16-8023-e5e904c78ebb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:24.435178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9533c-454a-4d16-8023-e5e904c78ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:24.435447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9533c-454a-4d16-8023-e5e904c78ebb None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:24.436176 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9533c-454a-4d16-8023-e5e904c78ebb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:24.436474 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9533c-454a-4d16-8023-e5e904c78ebb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:24.437983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9533c-454a-4d16-8023-e5e904c78ebb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:24.471586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 316/619] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Aug 9 18:30:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:24.478279 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9533c-454a-4d16-8023-e5e904c78ebb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:24.512159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32697a6-b641-45b3-9957-41c0ea871942 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:24.522180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:24.556390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a65912-4834-497a-bbe2-aac8fce81bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:24.557134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a65912-4834-497a-bbe2-aac8fce81bdb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:24.557413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a65912-4834-497a-bbe2-aac8fce81bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:24.557709 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a65912-4834-497a-bbe2-aac8fce81bdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:24.629756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:24.644088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9533c-454a-4d16-8023-e5e904c78ebb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:24.684770 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3d986-8414-4785-ab8e-1be2b09da1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:24.685421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3d986-8414-4785-ab8e-1be2b09da1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:24.685758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3d986-8414-4785-ab8e-1be2b09da1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:24.686033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3d986-8414-4785-ab8e-1be2b09da1f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:24.697800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3d986-8414-4785-ab8e-1be2b09da1f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:24.699327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3d986-8414-4785-ab8e-1be2b09da1f0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:24.699828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a65912-4834-497a-bbe2-aac8fce81bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:24.701656 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a65912-4834-497a-bbe2-aac8fce81bdb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-09b3d986-8414-4785-ab8e-1be2b09da1f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:24.716712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:24.719093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:24.721181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:24.721181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:24.721181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:24.725623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 317/620] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:30:24.725623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f14d56-525c-482e-a722-470871d3a5c5 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:24.725623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f14d56-525c-482e-a722-470871d3a5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:24.725623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f14d56-525c-482e-a722-470871d3a5c5 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:24.726826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 318/621] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:24.727339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f14d56-525c-482e-a722-470871d3a5c5 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-19a65912-4834-497a-bbe2-aac8fce81bdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:24.729722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:24.731836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:24.733330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:24.733330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:24.733330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:24.733330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 304/622] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:24.735277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36873cd3-a724-455e-a514-234f8f5adb7e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:24.735741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36873cd3-a724-455e-a514-234f8f5adb7e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:24.735899 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36873cd3-a724-455e-a514-234f8f5adb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:24.765618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-36873cd3-a724-455e-a514-234f8f5adb7e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:24.767078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-36873cd3-a724-455e-a514-234f8f5adb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:24.845172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f14d56-525c-482e-a722-470871d3a5c5 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:24.861684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 305/623] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:24.862662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36873cd3-a724-455e-a514-234f8f5adb7e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:24.907799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bb94c-5bfd-4d5e-bd32-6fe0d6d0f9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:24.908666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bb94c-5bfd-4d5e-bd32-6fe0d6d0f9ec None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:24.908945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bb94c-5bfd-4d5e-bd32-6fe0d6d0f9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:24.909214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bb94c-5bfd-4d5e-bd32-6fe0d6d0f9ec None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:24.910159 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bb94c-5bfd-4d5e-bd32-6fe0d6d0f9ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:24.910831 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bb94c-5bfd-4d5e-bd32-6fe0d6d0f9ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:24.912491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bb94c-5bfd-4d5e-bd32-6fe0d6d0f9ec None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:24.924741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:24.933972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 319/624] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:24.934805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bb94c-5bfd-4d5e-bd32-6fe0d6d0f9ec None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:24.980623 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:24.990762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36873cd3-a724-455e-a514-234f8f5adb7e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:25.037574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75080d0c-0cf0-45e3-b284-968b05239d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:25.039154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75080d0c-0cf0-45e3-b284-968b05239d21 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:25.040210 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75080d0c-0cf0-45e3-b284-968b05239d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:25.040692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75080d0c-0cf0-45e3-b284-968b05239d21 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:25.041746 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75080d0c-0cf0-45e3-b284-968b05239d21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:25.042509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75080d0c-0cf0-45e3-b284-968b05239d21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:25.044125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75080d0c-0cf0-45e3-b284-968b05239d21 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:25.049743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bb94c-5bfd-4d5e-bd32-6fe0d6d0f9ec None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:25.058805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 306/625] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:25.060134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75080d0c-0cf0-45e3-b284-968b05239d21 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:25.089753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5483d28-aa2c-4ba4-a29e-84f5528e0f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:25.090518 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5483d28-aa2c-4ba4-a29e-84f5528e0f5e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:25.090744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5483d28-aa2c-4ba4-a29e-84f5528e0f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:25.091020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5483d28-aa2c-4ba4-a29e-84f5528e0f5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:25.102584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5483d28-aa2c-4ba4-a29e-84f5528e0f5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-d5483d28-aa2c-4ba4-a29e-84f5528e0f5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:25.107911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:25.108903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:25.110338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:25.110338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:25.110338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:25.110338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:25.110338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:25.110338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:25.110338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:25.110338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:25.110338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:25.110338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:25.110338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:25.110338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:25.114020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 320/626] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:25.115021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5483d28-aa2c-4ba4-a29e-84f5528e0f5e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:25.175699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75080d0c-0cf0-45e3-b284-968b05239d21 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:25.188674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:25.212716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836943cb-2c1a-4e47-b096-4d7233ec44b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:25.213760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836943cb-2c1a-4e47-b096-4d7233ec44b9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:25.214254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836943cb-2c1a-4e47-b096-4d7233ec44b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:25.214751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836943cb-2c1a-4e47-b096-4d7233ec44b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:25.222286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836943cb-2c1a-4e47-b096-4d7233ec44b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:25.223988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836943cb-2c1a-4e47-b096-4d7233ec44b9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:25.237043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5483d28-aa2c-4ba4-a29e-84f5528e0f5e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-836943cb-2c1a-4e47-b096-4d7233ec44b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:25.237711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:25.238613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:25.239497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:25.239497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:25.239497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:25.239497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:25.239497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:25.239497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:25.239497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:25.239497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:25.241692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 307/627] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:25.242858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836943cb-2c1a-4e47-b096-4d7233ec44b9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:25.255762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:25.281910 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e136067-b646-4193-a5c8-f2980b42b6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:25.282865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e136067-b646-4193-a5c8-f2980b42b6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:25.283289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e136067-b646-4193-a5c8-f2980b42b6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:25.283701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e136067-b646-4193-a5c8-f2980b42b6a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:25.292489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e136067-b646-4193-a5c8-f2980b42b6a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:25.294188 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e136067-b646-4193-a5c8-f2980b42b6a2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-0e136067-b646-4193-a5c8-f2980b42b6a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:25.305874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:25.306871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:25.307978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:25.307978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:25.307978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:25.307978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:25.307978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:25.307978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:25.307978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:25.307978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:25.307978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:30:25.307978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:25.310087 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 321/628] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 9 18:30:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:25.311093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e136067-b646-4193-a5c8-f2980b42b6a2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:25.360194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836943cb-2c1a-4e47-b096-4d7233ec44b9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:25.401675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5e71d-0639-491d-b738-88a3766627ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:25.402447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5e71d-0639-491d-b738-88a3766627ae None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:25.402827 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5e71d-0639-491d-b738-88a3766627ae None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:25.403223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5e71d-0639-491d-b738-88a3766627ae None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:25.404146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5e71d-0639-491d-b738-88a3766627ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:25.404551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5e71d-0639-491d-b738-88a3766627ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:25.406053 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5e71d-0639-491d-b738-88a3766627ae None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:25.419874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 308/629] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:25.419874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5e71d-0639-491d-b738-88a3766627ae None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:25.427814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e136067-b646-4193-a5c8-f2980b42b6a2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:25.449853 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:25.468457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f0cf6-fc5d-467b-91f6-a32580df224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:25.469070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f0cf6-fc5d-467b-91f6-a32580df224c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:25.469408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f0cf6-fc5d-467b-91f6-a32580df224c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:25.469718 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f0cf6-fc5d-467b-91f6-a32580df224c None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:25.470577 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f0cf6-fc5d-467b-91f6-a32580df224c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:25.470888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f0cf6-fc5d-467b-91f6-a32580df224c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:25.472699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f0cf6-fc5d-467b-91f6-a32580df224c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:25.495758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 322/630] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:25.496705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f0cf6-fc5d-467b-91f6-a32580df224c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:25.498812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:25.539828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5e71d-0639-491d-b738-88a3766627ae None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:25.587830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea507b87-2545-4102-8036-fb998b2d272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:25.592132 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea507b87-2545-4102-8036-fb998b2d272c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:25.592669 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea507b87-2545-4102-8036-fb998b2d272c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:25.593157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea507b87-2545-4102-8036-fb998b2d272c None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:25.594194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea507b87-2545-4102-8036-fb998b2d272c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:25.594750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea507b87-2545-4102-8036-fb998b2d272c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:25.596488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea507b87-2545-4102-8036-fb998b2d272c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:25.616074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 309/631] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 9 18:30:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:25.617328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea507b87-2545-4102-8036-fb998b2d272c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:25.625879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f0cf6-fc5d-467b-91f6-a32580df224c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:25.667842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70dd29-f369-43d0-aa9c-39b79d74de87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:25.668835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70dd29-f369-43d0-aa9c-39b79d74de87 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:25.669176 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70dd29-f369-43d0-aa9c-39b79d74de87 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:25.669452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70dd29-f369-43d0-aa9c-39b79d74de87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:25.679847 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70dd29-f369-43d0-aa9c-39b79d74de87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:25.681299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70dd29-f369-43d0-aa9c-39b79d74de87 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-6b70dd29-f369-43d0-aa9c-39b79d74de87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:25.698960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:25.699920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:25.700771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:25.700771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:25.700771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:25.704813 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 323/632] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:25.705659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70dd29-f369-43d0-aa9c-39b79d74de87 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:25.736847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea507b87-2545-4102-8036-fb998b2d272c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:25.740968 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:25.766073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:25.772073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83d05e-5c80-47cb-a29c-45dae9cafd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:25.772731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83d05e-5c80-47cb-a29c-45dae9cafd76 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:25.773034 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83d05e-5c80-47cb-a29c-45dae9cafd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:25.773899 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83d05e-5c80-47cb-a29c-45dae9cafd76 None admin] PATH_INFO: `/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/2ff735ecfcdf49089b5cfeafc56048cf/roles/106b56dfde9d4c4f9f32ca4bec0b3e87` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:25.788820 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83d05e-5c80-47cb-a29c-45dae9cafd76 None admin] RBAC: Authorizing `identity:create_grant(project_id=abcccf694e7b4f66a7372e0ede416caf, user_id=2ff735ecfcdf49089b5cfeafc56048cf, role_id=106b56dfde9d4c4f9f32ca4bec0b3e87)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:25.790384 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83d05e-5c80-47cb-a29c-45dae9cafd76 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:25.816441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70dd29-f369-43d0-aa9c-39b79d74de87 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:25.851390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65ac39-a2d3-4d64-8a45-de45612fad4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:25.851986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65ac39-a2d3-4d64-8a45-de45612fad4e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:25.852266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65ac39-a2d3-4d64-8a45-de45612fad4e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:25.852538 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65ac39-a2d3-4d64-8a45-de45612fad4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:25.873806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 310/633] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:30:25 2022] PUT /identity/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/2ff735ecfcdf49089b5cfeafc56048cf/roles/106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:30:25.875325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cf751-1d40-45fd-b94c-2646a818ae21 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:25.875611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cf751-1d40-45fd-b94c-2646a818ae21 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:25.875883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cf751-1d40-45fd-b94c-2646a818ae21 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:25.877369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 311/634] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:25.877988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cf751-1d40-45fd-b94c-2646a818ae21 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:25.952855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65ac39-a2d3-4d64-8a45-de45612fad4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:25.954410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65ac39-a2d3-4d64-8a45-de45612fad4e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:25.999235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:26.016803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cf751-1d40-45fd-b94c-2646a818ae21 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:26.065370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892c0f1-63dc-467a-bac5-5ffddb39d37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:26.066127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892c0f1-63dc-467a-bac5-5ffddb39d37b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:26.066466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892c0f1-63dc-467a-bac5-5ffddb39d37b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:26.066744 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892c0f1-63dc-467a-bac5-5ffddb39d37b None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:26.067398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892c0f1-63dc-467a-bac5-5ffddb39d37b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:26.067704 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892c0f1-63dc-467a-bac5-5ffddb39d37b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:26.069072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892c0f1-63dc-467a-bac5-5ffddb39d37b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:26.081489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 312/635] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:26.082574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e899c-1f84-40f3-99c4-ec1c9a253c90 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:26.082910 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e899c-1f84-40f3-99c4-ec1c9a253c90 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:26.083187 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e899c-1f84-40f3-99c4-ec1c9a253c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:26.095866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 324/636] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:26.095866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf0c4a-0a31-4089-9b64-126894f35b0e None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:26.095866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf0c4a-0a31-4089-9b64-126894f35b0e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:26.095866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf0c4a-0a31-4089-9b64-126894f35b0e None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:26.095866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 325/637] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:26.097837 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf0c4a-0a31-4089-9b64-126894f35b0e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:26.105741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f50e899c-1f84-40f3-99c4-ec1c9a253c90 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:26.106941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f50e899c-1f84-40f3-99c4-ec1c9a253c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:26.148545 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:26.190941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 313/638] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:26.192834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e899c-1f84-40f3-99c4-ec1c9a253c90 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:26.232444 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf0c4a-0a31-4089-9b64-126894f35b0e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:26.277890 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:26.286550 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8c8b2-3aad-4045-a473-58f6394dd0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:26.289392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8c8b2-3aad-4045-a473-58f6394dd0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:26.289392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8c8b2-3aad-4045-a473-58f6394dd0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:26.289392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8c8b2-3aad-4045-a473-58f6394dd0e4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:26.296371 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8c8b2-3aad-4045-a473-58f6394dd0e4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:26.297794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8c8b2-3aad-4045-a473-58f6394dd0e4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-e6b8c8b2-3aad-4045-a473-58f6394dd0e4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:26.311746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:26.312759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:26.314135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:26.314135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:26.314135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:26.314135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:26.314135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:26.314135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:26.314135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:26.314135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:26.314135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 18:30:26.314135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:26.314135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 326/639] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:26.314135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5707e-e885-4638-bd79-c8d644a2c431 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:26.314897 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5707e-e885-4638-bd79-c8d644a2c431 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:26.314897 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5707e-e885-4638-bd79-c8d644a2c431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:26.335096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-62d5707e-e885-4638-bd79-c8d644a2c431 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:26.340678 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-62d5707e-e885-4638-bd79-c8d644a2c431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:26.352817 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e899c-1f84-40f3-99c4-ec1c9a253c90 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:26.396947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d896a01-a7b6-43a9-bd74-c13745ab4b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:26.397931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d896a01-a7b6-43a9-bd74-c13745ab4b19 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:26.398247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d896a01-a7b6-43a9-bd74-c13745ab4b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:26.398519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d896a01-a7b6-43a9-bd74-c13745ab4b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:26.418507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d896a01-a7b6-43a9-bd74-c13745ab4b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:26.429964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d896a01-a7b6-43a9-bd74-c13745ab4b19 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:26.430139 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 327/640] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:26.430139 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5707e-e885-4638-bd79-c8d644a2c431 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-6d896a01-a7b6-43a9-bd74-c13745ab4b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:26.451828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:26.453002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:26.454280 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:26.454280 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:26.454280 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:26.458472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 314/641] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:26.459534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d896a01-a7b6-43a9-bd74-c13745ab4b19 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:26.489106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:26.581228 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d896a01-a7b6-43a9-bd74-c13745ab4b19 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:26.624550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb847fd-c05f-46b3-92c3-57af6eb0b961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:26.625554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb847fd-c05f-46b3-92c3-57af6eb0b961 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:26.626072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb847fd-c05f-46b3-92c3-57af6eb0b961 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:26.626526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb847fd-c05f-46b3-92c3-57af6eb0b961 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:26.627461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb847fd-c05f-46b3-92c3-57af6eb0b961 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:26.628003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb847fd-c05f-46b3-92c3-57af6eb0b961 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:26.629771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb847fd-c05f-46b3-92c3-57af6eb0b961 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:26.651856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 315/642] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Aug 9 18:30:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:26.653158 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb847fd-c05f-46b3-92c3-57af6eb0b961 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:26.666732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5707e-e885-4638-bd79-c8d644a2c431 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:26.692448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:26.706086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8390622-f2df-43d4-ba22-2152db7910e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:26.706932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8390622-f2df-43d4-ba22-2152db7910e8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:26.707213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8390622-f2df-43d4-ba22-2152db7910e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:26.707485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8390622-f2df-43d4-ba22-2152db7910e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:26.719259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8390622-f2df-43d4-ba22-2152db7910e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-d8390622-f2df-43d4-ba22-2152db7910e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:26.725842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:26.727611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:26.728520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:26.728520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:26.728520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:26.728520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:26.728520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:26.728520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:26.728520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:26.728520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:26.728520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:26.728520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:26.728520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:26.728520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:26.730387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 328/643] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:26.731345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8390622-f2df-43d4-ba22-2152db7910e8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:26.781639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:26.802184 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb847fd-c05f-46b3-92c3-57af6eb0b961 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:26.840376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f44e3-8187-4a35-86da-25dfab5fc50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:26.840916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f44e3-8187-4a35-86da-25dfab5fc50e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:26.841190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f44e3-8187-4a35-86da-25dfab5fc50e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:26.841455 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f44e3-8187-4a35-86da-25dfab5fc50e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:26.854282 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f44e3-8187-4a35-86da-25dfab5fc50e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:26.854593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8390622-f2df-43d4-ba22-2152db7910e8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-386f44e3-8187-4a35-86da-25dfab5fc50e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:26.859126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:26.860039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:26.861121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 316/644] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:26.862206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f44e3-8187-4a35-86da-25dfab5fc50e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:26.896073 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb697b1e-2042-446f-866c-86b9be3f4143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:26.896678 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb697b1e-2042-446f-866c-86b9be3f4143 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:26.896952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb697b1e-2042-446f-866c-86b9be3f4143 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:26.897215 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb697b1e-2042-446f-866c-86b9be3f4143 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:26.897883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb697b1e-2042-446f-866c-86b9be3f4143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:26.898684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb697b1e-2042-446f-866c-86b9be3f4143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:26.900031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb697b1e-2042-446f-866c-86b9be3f4143 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:26.914261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 329/645] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:26.914261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb697b1e-2042-446f-866c-86b9be3f4143 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:26.961153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:26.979136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f44e3-8187-4a35-86da-25dfab5fc50e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:27.016382 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d40936-f677-42ed-9b72-e5321037096d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:27.016989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d40936-f677-42ed-9b72-e5321037096d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:27.017264 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d40936-f677-42ed-9b72-e5321037096d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:27.017563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d40936-f677-42ed-9b72-e5321037096d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:27.027912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb697b1e-2042-446f-866c-86b9be3f4143 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:27.028543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:27.065437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9eeb3-ec48-4c2e-9965-44a0c95e67ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:27.066361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9eeb3-ec48-4c2e-9965-44a0c95e67ac None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:27.066801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9eeb3-ec48-4c2e-9965-44a0c95e67ac None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:27.067076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9eeb3-ec48-4c2e-9965-44a0c95e67ac None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:27.067739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9eeb3-ec48-4c2e-9965-44a0c95e67ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:27.068043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9eeb3-ec48-4c2e-9965-44a0c95e67ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:27.069466 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9eeb3-ec48-4c2e-9965-44a0c95e67ac None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:27.085024 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 330/646] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:27.085024 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9eeb3-ec48-4c2e-9965-44a0c95e67ac None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:27.093593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d40936-f677-42ed-9b72-e5321037096d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:27.098545 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d40936-f677-42ed-9b72-e5321037096d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:27.192638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9eeb3-ec48-4c2e-9965-44a0c95e67ac None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:27.195717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:27.217625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 317/647] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:27.218474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d40936-f677-42ed-9b72-e5321037096d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:27.232717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704bb47-9d7a-415b-861e-1a035f276a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:27.233553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704bb47-9d7a-415b-861e-1a035f276a91 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:27.233990 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704bb47-9d7a-415b-861e-1a035f276a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:27.234423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704bb47-9d7a-415b-861e-1a035f276a91 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:27.235077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704bb47-9d7a-415b-861e-1a035f276a91 None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:27.235376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704bb47-9d7a-415b-861e-1a035f276a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:27.236713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704bb47-9d7a-415b-861e-1a035f276a91 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:27.245566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 331/648] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:27.250489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704bb47-9d7a-415b-861e-1a035f276a91 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:27.267472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:27.333652 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d40936-f677-42ed-9b72-e5321037096d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:27.367981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704bb47-9d7a-415b-861e-1a035f276a91 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:27.372199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815a4ac-ba10-40c9-a264-370f0e7e667b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:27.372816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815a4ac-ba10-40c9-a264-370f0e7e667b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:27.373091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815a4ac-ba10-40c9-a264-370f0e7e667b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:27.373357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815a4ac-ba10-40c9-a264-370f0e7e667b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:27.380598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815a4ac-ba10-40c9-a264-370f0e7e667b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:27.382691 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815a4ac-ba10-40c9-a264-370f0e7e667b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-7815a4ac-ba10-40c9-a264-370f0e7e667b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:27.394065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:27.395019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:27.395952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:27.395952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:27.395952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:27.395952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:27.395952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:27.395952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:27.395952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:27.395952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:27.395952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 318/649] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:27.395952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815a4ac-ba10-40c9-a264-370f0e7e667b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:27.408871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3609c0-7e56-4bd5-a7f5-985d5b2c809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:27.409647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3609c0-7e56-4bd5-a7f5-985d5b2c809f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:27.410129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3609c0-7e56-4bd5-a7f5-985d5b2c809f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:27.410238 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3609c0-7e56-4bd5-a7f5-985d5b2c809f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:27.424133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3609c0-7e56-4bd5-a7f5-985d5b2c809f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:27.425603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3609c0-7e56-4bd5-a7f5-985d5b2c809f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-7d3609c0-7e56-4bd5-a7f5-985d5b2c809f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:27.439874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:27.441262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:27.442246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:27.442246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:27.442246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:27.445625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 332/650] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:27.446497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3609c0-7e56-4bd5-a7f5-985d5b2c809f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:27.466016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:27.506002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:27.509021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815a4ac-ba10-40c9-a264-370f0e7e667b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:27.547649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746cc86d-9ccf-4eab-96e8-2a64e6120b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:27.548277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cc86d-9ccf-4eab-96e8-2a64e6120b37 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:27.548557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cc86d-9ccf-4eab-96e8-2a64e6120b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:27.548845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cc86d-9ccf-4eab-96e8-2a64e6120b37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:27.556979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3609c0-7e56-4bd5-a7f5-985d5b2c809f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:27.559880 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746cc86d-9ccf-4eab-96e8-2a64e6120b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:27.560936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746cc86d-9ccf-4eab-96e8-2a64e6120b37 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-746cc86d-9ccf-4eab-96e8-2a64e6120b37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:27.575685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:27.576631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:27.582915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:27.582915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:27.582915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:27.582915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 319/651] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:27.582915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746cc86d-9ccf-4eab-96e8-2a64e6120b37 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:27.598085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe09dd1-7395-4007-b17d-4deae6f30dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:27.598811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe09dd1-7395-4007-b17d-4deae6f30dfb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:27.599093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe09dd1-7395-4007-b17d-4deae6f30dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:27.599364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe09dd1-7395-4007-b17d-4deae6f30dfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:27.610492 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe09dd1-7395-4007-b17d-4deae6f30dfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:27.612006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe09dd1-7395-4007-b17d-4deae6f30dfb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-3fe09dd1-7395-4007-b17d-4deae6f30dfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:27.628411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:27.629501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:27.630500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:27.630500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:27.630500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:27.634203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 333/652] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:27.635051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe09dd1-7395-4007-b17d-4deae6f30dfb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:27.692924 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746cc86d-9ccf-4eab-96e8-2a64e6120b37 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:27.706253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:27.733908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af83ace-97b6-4b24-94a6-53ff2750b04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:27.735068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af83ace-97b6-4b24-94a6-53ff2750b04d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:27.735411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af83ace-97b6-4b24-94a6-53ff2750b04d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:27.735753 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af83ace-97b6-4b24-94a6-53ff2750b04d None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:27.736523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af83ace-97b6-4b24-94a6-53ff2750b04d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:27.736915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af83ace-97b6-4b24-94a6-53ff2750b04d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:27.738440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af83ace-97b6-4b24-94a6-53ff2750b04d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:27.739972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe09dd1-7395-4007-b17d-4deae6f30dfb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:27.756212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 320/653] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:27.756212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af83ace-97b6-4b24-94a6-53ff2750b04d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:27.769327 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:27.786887 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3456cc-a49e-4ace-a5e9-8f373edade1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:27.787491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3456cc-a49e-4ace-a5e9-8f373edade1e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:27.787775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3456cc-a49e-4ace-a5e9-8f373edade1e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:27.788045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3456cc-a49e-4ace-a5e9-8f373edade1e None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:27.788738 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3456cc-a49e-4ace-a5e9-8f373edade1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:27.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3456cc-a49e-4ace-a5e9-8f373edade1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:27.790453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3456cc-a49e-4ace-a5e9-8f373edade1e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:27.802658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 334/654] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:27.807463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3456cc-a49e-4ace-a5e9-8f373edade1e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:27.869154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af83ace-97b6-4b24-94a6-53ff2750b04d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:27.905734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c2c36-8bfc-4a25-b8e0-57c7e5e8e729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:27.906253 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c2c36-8bfc-4a25-b8e0-57c7e5e8e729 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:27.906528 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c2c36-8bfc-4a25-b8e0-57c7e5e8e729 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:27.906938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c2c36-8bfc-4a25-b8e0-57c7e5e8e729 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:27.907631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c2c36-8bfc-4a25-b8e0-57c7e5e8e729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:27.907958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c2c36-8bfc-4a25-b8e0-57c7e5e8e729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:27.909306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c2c36-8bfc-4a25-b8e0-57c7e5e8e729 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:27.917779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3456cc-a49e-4ace-a5e9-8f373edade1e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:27.925332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 321/655] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:27.925332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c2c36-8bfc-4a25-b8e0-57c7e5e8e729 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:27.957384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:27.963880 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee32a21-931e-426e-a80a-1c3d10d9f0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:27.964488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee32a21-931e-426e-a80a-1c3d10d9f0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:27.970229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee32a21-931e-426e-a80a-1c3d10d9f0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:27.970229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee32a21-931e-426e-a80a-1c3d10d9f0a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:27.970229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee32a21-931e-426e-a80a-1c3d10d9f0a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:27.970229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee32a21-931e-426e-a80a-1c3d10d9f0a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:27.970229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee32a21-931e-426e-a80a-1c3d10d9f0a3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:27.978418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 335/656] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:27.979211 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee32a21-931e-426e-a80a-1c3d10d9f0a3 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:28.001303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:28.038265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c2c36-8bfc-4a25-b8e0-57c7e5e8e729 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:28.073509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97578b85-85c5-4530-9fab-87f52a12a739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:28.074138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97578b85-85c5-4530-9fab-87f52a12a739 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:28.074415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97578b85-85c5-4530-9fab-87f52a12a739 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:28.074687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97578b85-85c5-4530-9fab-87f52a12a739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:28.075413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97578b85-85c5-4530-9fab-87f52a12a739 None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=e2ecd7d1cce644b0982cc23740537933) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:28.075741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97578b85-85c5-4530-9fab-87f52a12a739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:28.077157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97578b85-85c5-4530-9fab-87f52a12a739 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:28.084094 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 322/657] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:27 2022] GET /identity/v3/role_assignments?user.id=e2ecd7d1cce644b0982cc23740537933&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:28.084893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97578b85-85c5-4530-9fab-87f52a12a739 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:28.095898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee32a21-931e-426e-a80a-1c3d10d9f0a3 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:28.135418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ebaa9-ecba-4068-acdc-80e175335e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:28.136061 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ebaa9-ecba-4068-acdc-80e175335e56 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:28.136342 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ebaa9-ecba-4068-acdc-80e175335e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:28.136617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ebaa9-ecba-4068-acdc-80e175335e56 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:28.145492 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ebaa9-ecba-4068-acdc-80e175335e56 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:28.147011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ebaa9-ecba-4068-acdc-80e175335e56 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-025ebaa9-ecba-4068-acdc-80e175335e56 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:28.159368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:28.160348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:28.161265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:28.161265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:28.161265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:28.161265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:28.161265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:28.161265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:28.161265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:28.161265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:28.161265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 18:30:28.161265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:28.161265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 336/658] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:28.162305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1acfb-ff1a-4c06-b17f-806ec4957b5e None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:28.162305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1acfb-ff1a-4c06-b17f-806ec4957b5e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:28.162305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1acfb-ff1a-4c06-b17f-806ec4957b5e None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:28.167944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 337/659] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:28.168851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1acfb-ff1a-4c06-b17f-806ec4957b5e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:28.217726 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97578b85-85c5-4530-9fab-87f52a12a739 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:28.231108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:28.258982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54653767-0e4a-4f73-9767-3de0df8ed2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:28.259615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54653767-0e4a-4f73-9767-3de0df8ed2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:28.260578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54653767-0e4a-4f73-9767-3de0df8ed2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:28.260578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54653767-0e4a-4f73-9767-3de0df8ed2c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:28.263126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:28.268201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54653767-0e4a-4f73-9767-3de0df8ed2c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:28.269982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54653767-0e4a-4f73-9767-3de0df8ed2c9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-54653767-0e4a-4f73-9767-3de0df8ed2c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:28.281906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:28.282794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:28.283650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:28.283650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:28.283650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:28.283650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:28.283650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:28.283650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:28.283650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:28.283650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:28.283650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 323/660] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:28.285490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54653767-0e4a-4f73-9767-3de0df8ed2c9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:28.298852 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1acfb-ff1a-4c06-b17f-806ec4957b5e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:28.348435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d470180-ea43-40b1-af19-af8789c0a62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:28.349058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d470180-ea43-40b1-af19-af8789c0a62a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:28.349333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d470180-ea43-40b1-af19-af8789c0a62a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:28.349627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d470180-ea43-40b1-af19-af8789c0a62a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:28.357146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d470180-ea43-40b1-af19-af8789c0a62a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:28.362971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d470180-ea43-40b1-af19-af8789c0a62a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-8d470180-ea43-40b1-af19-af8789c0a62a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:28.375934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:28.376941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:28.377929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:28.377929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:28.377929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:28.377929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:28.377929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:28.377929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:28.377929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:28.377929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:28.377929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 18:30:28.377929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:28.377929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 338/661] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:28.379092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda86753-ea6d-4657-8325-e7bf43f75180 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:28.379092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda86753-ea6d-4657-8325-e7bf43f75180 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:28.379339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda86753-ea6d-4657-8325-e7bf43f75180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:28.401850 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-eda86753-ea6d-4657-8325-e7bf43f75180 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:28.403059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-eda86753-ea6d-4657-8325-e7bf43f75180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:28.406176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54653767-0e4a-4f73-9767-3de0df8ed2c9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:28.444717 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93279261-fb6a-4f87-9f4f-ed2f2a1cb156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:28.445340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93279261-fb6a-4f87-9f4f-ed2f2a1cb156 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:28.445723 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93279261-fb6a-4f87-9f4f-ed2f2a1cb156 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:28.446066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93279261-fb6a-4f87-9f4f-ed2f2a1cb156 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:28.446864 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93279261-fb6a-4f87-9f4f-ed2f2a1cb156 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:28.447242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93279261-fb6a-4f87-9f4f-ed2f2a1cb156 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:28.448737 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93279261-fb6a-4f87-9f4f-ed2f2a1cb156 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:28.467106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 324/662] 10.222.0.57 () {56 vars in 1137 bytes} [Tue Aug 9 18:30:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:28.468051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93279261-fb6a-4f87-9f4f-ed2f2a1cb156 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:28.481488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 339/663] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:28.482937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda86753-ea6d-4657-8325-e7bf43f75180 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:28.533086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:28.539178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:28.584371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93279261-fb6a-4f87-9f4f-ed2f2a1cb156 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:28.593094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda86753-ea6d-4657-8325-e7bf43f75180 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:28.624207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c39355-9b4f-4f25-b6ac-3abc50e0f13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:28.624948 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c39355-9b4f-4f25-b6ac-3abc50e0f13b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:28.625274 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c39355-9b4f-4f25-b6ac-3abc50e0f13b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:28.625646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c39355-9b4f-4f25-b6ac-3abc50e0f13b None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:28.626576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c39355-9b4f-4f25-b6ac-3abc50e0f13b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:28.626945 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c39355-9b4f-4f25-b6ac-3abc50e0f13b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:28.628435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c39355-9b4f-4f25-b6ac-3abc50e0f13b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:28.631991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad8e70-71df-43b3-8e40-8892425483a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:28.632714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad8e70-71df-43b3-8e40-8892425483a2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:28.633062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad8e70-71df-43b3-8e40-8892425483a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:28.633402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad8e70-71df-43b3-8e40-8892425483a2 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:28.634227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad8e70-71df-43b3-8e40-8892425483a2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:28.634603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad8e70-71df-43b3-8e40-8892425483a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:28.636084 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad8e70-71df-43b3-8e40-8892425483a2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:28.642668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 325/664] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:28.643489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c39355-9b4f-4f25-b6ac-3abc50e0f13b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:28.659042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 340/665] 10.222.0.57 () {56 vars in 1139 bytes} [Tue Aug 9 18:30:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:28.659885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad8e70-71df-43b3-8e40-8892425483a2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:28.758760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c39355-9b4f-4f25-b6ac-3abc50e0f13b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:28.761576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:28.765587 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad8e70-71df-43b3-8e40-8892425483a2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:28.773236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:28.795251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60ac56-75da-47a9-bba8-efda164fada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:28.796138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60ac56-75da-47a9-bba8-efda164fada6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:28.796574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60ac56-75da-47a9-bba8-efda164fada6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:28.797005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60ac56-75da-47a9-bba8-efda164fada6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:28.802576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f7de-dd5d-47e8-9313-c5b40f547855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:28.806576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f7de-dd5d-47e8-9313-c5b40f547855 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:28.806576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f7de-dd5d-47e8-9313-c5b40f547855 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:28.806576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f7de-dd5d-47e8-9313-c5b40f547855 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:28.818529 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8f7de-dd5d-47e8-9313-c5b40f547855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-9dd8f7de-dd5d-47e8-9313-c5b40f547855 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:28.823653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:28.824804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:28.825819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:28.825819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:28.825819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:28.825819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:28.825819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:28.825819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:28.825819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:28.825819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:28.825819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:28.825819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:28.825819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:28.825819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:28.826487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 341/666] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:28.826790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f7de-dd5d-47e8-9313-c5b40f547855 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:28.880449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60ac56-75da-47a9-bba8-efda164fada6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:28.882464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60ac56-75da-47a9-bba8-efda164fada6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:28.933378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f7de-dd5d-47e8-9313-c5b40f547855 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:28.970228 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43434978-9308-4afe-bc50-3fcb86e47f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:28.970909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43434978-9308-4afe-bc50-3fcb86e47f55 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:28.971447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43434978-9308-4afe-bc50-3fcb86e47f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:28.971447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43434978-9308-4afe-bc50-3fcb86e47f55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:28.972186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43434978-9308-4afe-bc50-3fcb86e47f55 None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=77300e25b9004a598f8f7f12f3cf4866) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:28.972499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43434978-9308-4afe-bc50-3fcb86e47f55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:28.973963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43434978-9308-4afe-bc50-3fcb86e47f55 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:28.989567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 342/667] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:28 2022] GET /identity/v3/role_assignments?user.id=77300e25b9004a598f8f7f12f3cf4866&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:28.989567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43434978-9308-4afe-bc50-3fcb86e47f55 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:29.007575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 326/668] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:29.008536 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60ac56-75da-47a9-bba8-efda164fada6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:29.024461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:29.026094 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:29.111673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43434978-9308-4afe-bc50-3fcb86e47f55 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:29.130066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60ac56-75da-47a9-bba8-efda164fada6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:29.154943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a65c6-b924-4124-b71d-f14076c010ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:29.155560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a65c6-b924-4124-b71d-f14076c010ba None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:29.156442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a65c6-b924-4124-b71d-f14076c010ba None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:29.156442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a65c6-b924-4124-b71d-f14076c010ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:29.186921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793c9f6-6b6f-4e1b-8dad-8677e1a45ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:29.187737 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793c9f6-6b6f-4e1b-8dad-8677e1a45ace None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:29.188194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793c9f6-6b6f-4e1b-8dad-8677e1a45ace None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:29.189325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793c9f6-6b6f-4e1b-8dad-8677e1a45ace None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:29.190156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793c9f6-6b6f-4e1b-8dad-8677e1a45ace None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:29.190567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793c9f6-6b6f-4e1b-8dad-8677e1a45ace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:29.192931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793c9f6-6b6f-4e1b-8dad-8677e1a45ace None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:29.204793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 327/669] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:29.206078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793c9f6-6b6f-4e1b-8dad-8677e1a45ace None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:29.251749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a65c6-b924-4124-b71d-f14076c010ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:29.253221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a65c6-b924-4124-b71d-f14076c010ba None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:29.292177 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:29.311096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:29.337924 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793c9f6-6b6f-4e1b-8dad-8677e1a45ace None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:29.386742 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac81f63-fbca-458a-bfd1-0341b1f1e08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:29.387384 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac81f63-fbca-458a-bfd1-0341b1f1e08c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:29.387668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac81f63-fbca-458a-bfd1-0341b1f1e08c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:29.388071 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac81f63-fbca-458a-bfd1-0341b1f1e08c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:29.397084 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 343/670] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:29.398939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a65c6-b924-4124-b71d-f14076c010ba None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:29.404105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac81f63-fbca-458a-bfd1-0341b1f1e08c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:29.405598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac81f63-fbca-458a-bfd1-0341b1f1e08c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-dac81f63-fbca-458a-bfd1-0341b1f1e08c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:29.423125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:29.424103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:29.425001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:29.425001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:29.425001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:29.429759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 328/671] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:29.430639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac81f63-fbca-458a-bfd1-0341b1f1e08c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:29.532416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a65c6-b924-4124-b71d-f14076c010ba None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:29.558495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:29.565916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac81f63-fbca-458a-bfd1-0341b1f1e08c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:29.570927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a229c-ea53-4382-be4d-9b3d913a7747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:29.574340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a229c-ea53-4382-be4d-9b3d913a7747 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:29.574340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a229c-ea53-4382-be4d-9b3d913a7747 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:29.574340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a229c-ea53-4382-be4d-9b3d913a7747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:29.587490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a229c-ea53-4382-be4d-9b3d913a7747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:29.589510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a229c-ea53-4382-be4d-9b3d913a7747 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:29.600846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:29.609275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc0539-f8f7-4d98-a6a0-4d95bd284377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:29.609934 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc0539-f8f7-4d98-a6a0-4d95bd284377 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:29.613943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc0539-f8f7-4d98-a6a0-4d95bd284377 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:29.613943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc0539-f8f7-4d98-a6a0-4d95bd284377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-d40a229c-ea53-4382-be4d-9b3d913a7747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:29.614121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:29.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:29.615921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:29.615921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:29.615921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:29.617176 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 344/672] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:29.618394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a229c-ea53-4382-be4d-9b3d913a7747 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:29.625936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc0539-f8f7-4d98-a6a0-4d95bd284377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:29.627400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc0539-f8f7-4d98-a6a0-4d95bd284377 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-31dc0539-f8f7-4d98-a6a0-4d95bd284377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:29.643237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:29.644206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:29.645073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:29.645073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:29.645073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:29.649459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 329/673] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:29.650354 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc0539-f8f7-4d98-a6a0-4d95bd284377 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:29.737311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a229c-ea53-4382-be4d-9b3d913a7747 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:29.760837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc0539-f8f7-4d98-a6a0-4d95bd284377 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:29.775699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f8c75-6fbe-467b-a3f5-13d5a2ab375c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:29.776715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f8c75-6fbe-467b-a3f5-13d5a2ab375c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:29.777216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f8c75-6fbe-467b-a3f5-13d5a2ab375c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:29.777739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f8c75-6fbe-467b-a3f5-13d5a2ab375c None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:29.778685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f8c75-6fbe-467b-a3f5-13d5a2ab375c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:29.779314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f8c75-6fbe-467b-a3f5-13d5a2ab375c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:29.780958 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f8c75-6fbe-467b-a3f5-13d5a2ab375c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:29.801147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 345/674] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:29.802391 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f8c75-6fbe-467b-a3f5-13d5a2ab375c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:29.806277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9d21f-0038-4e0c-b967-b27ffbd4b031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:29.806898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9d21f-0038-4e0c-b967-b27ffbd4b031 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:29.807179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9d21f-0038-4e0c-b967-b27ffbd4b031 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:29.807453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9d21f-0038-4e0c-b967-b27ffbd4b031 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:29.808114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9d21f-0038-4e0c-b967-b27ffbd4b031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:29.808420 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9d21f-0038-4e0c-b967-b27ffbd4b031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:29.809965 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9d21f-0038-4e0c-b967-b27ffbd4b031 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:29.828351 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 330/675] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:29.829400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9d21f-0038-4e0c-b967-b27ffbd4b031 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:29.845565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:29.874962 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:29.921555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f8c75-6fbe-467b-a3f5-13d5a2ab375c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:29.942734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9d21f-0038-4e0c-b967-b27ffbd4b031 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:29.977643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e31376-3128-4d10-9a4d-ccfdc544cfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:29.977643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e31376-3128-4d10-9a4d-ccfdc544cfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:29.977643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e31376-3128-4d10-9a4d-ccfdc544cfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:29.977643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e31376-3128-4d10-9a4d-ccfdc544cfa3 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:29.978069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e31376-3128-4d10-9a4d-ccfdc544cfa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:29.978069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e31376-3128-4d10-9a4d-ccfdc544cfa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:29.978069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e31376-3128-4d10-9a4d-ccfdc544cfa3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:29.983939 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c2367-6c8c-42d3-9ee2-008dd566d64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:29.985145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c2367-6c8c-42d3-9ee2-008dd566d64a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:29.985145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c2367-6c8c-42d3-9ee2-008dd566d64a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:29.985366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c2367-6c8c-42d3-9ee2-008dd566d64a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:29.992301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c2367-6c8c-42d3-9ee2-008dd566d64a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:29.993755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c2367-6c8c-42d3-9ee2-008dd566d64a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:30.004277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 346/676] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:30.004277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e31376-3128-4d10-9a4d-ccfdc544cfa3 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-065c2367-6c8c-42d3-9ee2-008dd566d64a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:30.014378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:30.023077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:30.026423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:30.026423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:30.026423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:30.026423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:30.026423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:30.026423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:30.026423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:30.026423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:30.026423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 331/677] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:30.026423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c2367-6c8c-42d3-9ee2-008dd566d64a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:30.139883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:30.144645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:30.147801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c2367-6c8c-42d3-9ee2-008dd566d64a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:30.154716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e31376-3128-4d10-9a4d-ccfdc544cfa3 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:30.190169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63687953-84b5-4238-a53f-f7a2fb8746fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:30.191561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63687953-84b5-4238-a53f-f7a2fb8746fa None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:30.191561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63687953-84b5-4238-a53f-f7a2fb8746fa None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:30.191561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63687953-84b5-4238-a53f-f7a2fb8746fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:30.193609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799109ed-2dcc-4ae0-9bd8-dd2a2a41c536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:30.195612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799109ed-2dcc-4ae0-9bd8-dd2a2a41c536 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:30.196101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799109ed-2dcc-4ae0-9bd8-dd2a2a41c536 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:30.196571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799109ed-2dcc-4ae0-9bd8-dd2a2a41c536 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:30.199847 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63687953-84b5-4238-a53f-f7a2fb8746fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:30.201761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63687953-84b5-4238-a53f-f7a2fb8746fa None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:30.206378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799109ed-2dcc-4ae0-9bd8-dd2a2a41c536 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:30.207864 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799109ed-2dcc-4ae0-9bd8-dd2a2a41c536 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-63687953-84b5-4238-a53f-f7a2fb8746fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:30.219028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:30.220209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:30.221265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:30.221265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:30.221265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:30.221265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:30.221265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:30.221265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:30.221265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:30.221265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:30.221265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 18:30:30.221265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:30.221265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 347/678] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:30.221265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63687953-84b5-4238-a53f-f7a2fb8746fa None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-799109ed-2dcc-4ae0-9bd8-dd2a2a41c536 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:30.227343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:30.228208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:30.229062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:30.229062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:30.229062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:30.229062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:30.229062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:30.229062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:30.229062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:30.229062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:30.233417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 332/679] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:30.234161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f704ad-cbc5-4409-ba8b-0b2c6c1d4055 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:30.234444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f704ad-cbc5-4409-ba8b-0b2c6c1d4055 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:30.235050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f704ad-cbc5-4409-ba8b-0b2c6c1d4055 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:30.237774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 333/680] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:30.237774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f704ad-cbc5-4409-ba8b-0b2c6c1d4055 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:30.341515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63687953-84b5-4238-a53f-f7a2fb8746fa None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:30.359096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f704ad-cbc5-4409-ba8b-0b2c6c1d4055 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:30.375817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949b454-a472-4bbe-b3fe-979105025e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:30.376419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949b454-a472-4bbe-b3fe-979105025e26 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:30.377079 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949b454-a472-4bbe-b3fe-979105025e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:30.377356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949b454-a472-4bbe-b3fe-979105025e26 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:30.378088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949b454-a472-4bbe-b3fe-979105025e26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:30.378389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949b454-a472-4bbe-b3fe-979105025e26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:30.379814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949b454-a472-4bbe-b3fe-979105025e26 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:30.392829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 348/681] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:30.393295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949b454-a472-4bbe-b3fe-979105025e26 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:30.404552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f32d1-b536-4596-a7c9-b5b001957dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:30.405162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f32d1-b536-4596-a7c9-b5b001957dcd None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:30.405444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f32d1-b536-4596-a7c9-b5b001957dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:30.405763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f32d1-b536-4596-a7c9-b5b001957dcd None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:30.406510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f32d1-b536-4596-a7c9-b5b001957dcd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:30.406820 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f32d1-b536-4596-a7c9-b5b001957dcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:30.408242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f32d1-b536-4596-a7c9-b5b001957dcd None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:30.410789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:30.427889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 334/682] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Aug 9 18:30:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:30.434131 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92d1e0-58b9-4528-bb2a-619c74dc137e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:30.434131 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92d1e0-58b9-4528-bb2a-619c74dc137e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:30.434131 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92d1e0-58b9-4528-bb2a-619c74dc137e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:30.453846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9e92d1e0-58b9-4528-bb2a-619c74dc137e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:30.455164 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9e92d1e0-58b9-4528-bb2a-619c74dc137e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:30.474546 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:30.516489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949b454-a472-4bbe-b3fe-979105025e26 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:30.531325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 335/683] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:30.532273 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92d1e0-58b9-4528-bb2a-619c74dc137e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:30.557058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497805c-2c5e-4f60-b17a-6526fa577292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:30.557716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497805c-2c5e-4f60-b17a-6526fa577292 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:30.557977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497805c-2c5e-4f60-b17a-6526fa577292 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:30.558252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497805c-2c5e-4f60-b17a-6526fa577292 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:30.558936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5497805c-2c5e-4f60-b17a-6526fa577292 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:30.559242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5497805c-2c5e-4f60-b17a-6526fa577292 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:30.560715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5497805c-2c5e-4f60-b17a-6526fa577292 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:30.573550 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 349/684] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:30.574289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497805c-2c5e-4f60-b17a-6526fa577292 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:30.649794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92d1e0-58b9-4528-bb2a-619c74dc137e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:30.660520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:30.687640 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443d44-a978-49a9-8082-a509cee69d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:30.688340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443d44-a978-49a9-8082-a509cee69d2a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:30.688687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443d44-a978-49a9-8082-a509cee69d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:30.688973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497805c-2c5e-4f60-b17a-6526fa577292 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:30.689078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443d44-a978-49a9-8082-a509cee69d2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:30.689950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0443d44-a978-49a9-8082-a509cee69d2a None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=5a58f9101f2d4785b96f0cc55be4330b) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:30.690454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0443d44-a978-49a9-8082-a509cee69d2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:30.691851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0443d44-a978-49a9-8082-a509cee69d2a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:30.698547 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 336/685] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:30 2022] GET /identity/v3/role_assignments?user.id=5a58f9101f2d4785b96f0cc55be4330b&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:30.699319 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443d44-a978-49a9-8082-a509cee69d2a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:30.719311 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:30.753215 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140dcc1-5041-4424-9f44-4ac9296c6c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:30.754165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140dcc1-5041-4424-9f44-4ac9296c6c18 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:30.754580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140dcc1-5041-4424-9f44-4ac9296c6c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:30.755068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140dcc1-5041-4424-9f44-4ac9296c6c18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:30.843961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443d44-a978-49a9-8082-a509cee69d2a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:30.849874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140dcc1-5041-4424-9f44-4ac9296c6c18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:30.851797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140dcc1-5041-4424-9f44-4ac9296c6c18 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:30.883030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0c76a-dbe7-4de4-b772-9736d1781f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:30.883769 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0c76a-dbe7-4de4-b772-9736d1781f56 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:30.884038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0c76a-dbe7-4de4-b772-9736d1781f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:30.884323 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0c76a-dbe7-4de4-b772-9736d1781f56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:30.900510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0c76a-dbe7-4de4-b772-9736d1781f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-09f0c76a-dbe7-4de4-b772-9736d1781f56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:30.904713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:30.905867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:30.906851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 337/686] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:30.907679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0c76a-dbe7-4de4-b772-9736d1781f56 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:30.958427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:30.997461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 350/687] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:30.999386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140dcc1-5041-4424-9f44-4ac9296c6c18 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:31.017665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:31.049838 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0c76a-dbe7-4de4-b772-9736d1781f56 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:31.089253 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e83c5-b2a7-4cb6-8aad-681114372cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:31.089889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e83c5-b2a7-4cb6-8aad-681114372cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:31.090166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e83c5-b2a7-4cb6-8aad-681114372cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:31.090417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e83c5-b2a7-4cb6-8aad-681114372cb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:31.091145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e83c5-b2a7-4cb6-8aad-681114372cb6 None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=88a7cdd610f74330a5da69c0de8472db) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:31.091448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e83c5-b2a7-4cb6-8aad-681114372cb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:31.092918 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e83c5-b2a7-4cb6-8aad-681114372cb6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:31.102082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 338/688] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:30 2022] GET /identity/v3/role_assignments?user.id=88a7cdd610f74330a5da69c0de8472db&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:31.103001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e83c5-b2a7-4cb6-8aad-681114372cb6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:31.140946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140dcc1-5041-4424-9f44-4ac9296c6c18 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:31.197401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e0b58-d547-47a6-8555-d0d6b684118e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:31.197999 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e0b58-d547-47a6-8555-d0d6b684118e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:31.198279 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e0b58-d547-47a6-8555-d0d6b684118e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:31.198551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e0b58-d547-47a6-8555-d0d6b684118e None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:31.199221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e0b58-d547-47a6-8555-d0d6b684118e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:31.199530 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e0b58-d547-47a6-8555-d0d6b684118e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:31.201124 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e0b58-d547-47a6-8555-d0d6b684118e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:31.211578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 351/689] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:31.212974 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad806709-1906-4391-b950-fb1af8213e0b None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:31.213263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad806709-1906-4391-b950-fb1af8213e0b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:31.213550 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad806709-1906-4391-b950-fb1af8213e0b None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:31.219090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 352/690] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:31.220163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad806709-1906-4391-b950-fb1af8213e0b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:31.248141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e83c5-b2a7-4cb6-8aad-681114372cb6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:31.281382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:31.309361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7a5da-a7a4-443e-858f-0199a1c40a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:31.310379 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7a5da-a7a4-443e-858f-0199a1c40a74 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:31.310798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7a5da-a7a4-443e-858f-0199a1c40a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:31.311199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7a5da-a7a4-443e-858f-0199a1c40a74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:31.314895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:31.325196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7a5da-a7a4-443e-858f-0199a1c40a74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:31.326884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7a5da-a7a4-443e-858f-0199a1c40a74 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-78e7a5da-a7a4-443e-858f-0199a1c40a74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:31.342117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:31.343145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:31.344038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:31.344038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:31.344038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:31.344038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 339/691] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:31.349578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1fae4-8518-41f3-a0db-6be59548f9c9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:31.349578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1fae4-8518-41f3-a0db-6be59548f9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:31.349578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1fae4-8518-41f3-a0db-6be59548f9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:31.371709 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-86d1fae4-8518-41f3-a0db-6be59548f9c9 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:31.374078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-86d1fae4-8518-41f3-a0db-6be59548f9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:31.379106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad806709-1906-4391-b950-fb1af8213e0b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:31.428635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde04852-c912-467b-bfce-b83497dd3368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:31.430416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde04852-c912-467b-bfce-b83497dd3368 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:31.431327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde04852-c912-467b-bfce-b83497dd3368 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:31.432051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde04852-c912-467b-bfce-b83497dd3368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:31.446370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde04852-c912-467b-bfce-b83497dd3368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:31.449212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde04852-c912-467b-bfce-b83497dd3368 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:31.465600 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 340/692] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:31.466628 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1fae4-8518-41f3-a0db-6be59548f9c9 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-dde04852-c912-467b-bfce-b83497dd3368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:31.468647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:31.469626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:31.470499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:31.470499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:31.470499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:31.473003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 353/693] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:31.484705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde04852-c912-467b-bfce-b83497dd3368 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:31.592138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1fae4-8518-41f3-a0db-6be59548f9c9 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:31.605737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:31.611846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:31.624549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde04852-c912-467b-bfce-b83497dd3368 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:31.630065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee0700-6af2-4859-be16-f42d25a04710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:31.630780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee0700-6af2-4859-be16-f42d25a04710 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:31.631136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee0700-6af2-4859-be16-f42d25a04710 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:31.631482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee0700-6af2-4859-be16-f42d25a04710 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:31.632234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee0700-6af2-4859-be16-f42d25a04710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:31.632612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee0700-6af2-4859-be16-f42d25a04710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:31.634099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee0700-6af2-4859-be16-f42d25a04710 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:31.650589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 341/694] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:31.651557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee0700-6af2-4859-be16-f42d25a04710 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:31.673966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf527b6-302e-4964-aa67-5f4f48ac11df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:31.675194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf527b6-302e-4964-aa67-5f4f48ac11df None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:31.675471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf527b6-302e-4964-aa67-5f4f48ac11df None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:31.675735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf527b6-302e-4964-aa67-5f4f48ac11df None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:31.676382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf527b6-302e-4964-aa67-5f4f48ac11df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:31.676680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf527b6-302e-4964-aa67-5f4f48ac11df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:31.680814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf527b6-302e-4964-aa67-5f4f48ac11df None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:31.701619 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 354/695] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:30:31.701619 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf527b6-302e-4964-aa67-5f4f48ac11df None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:31.767371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee0700-6af2-4859-be16-f42d25a04710 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:31.807143 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf527b6-302e-4964-aa67-5f4f48ac11df None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:31.809841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdecf3-4ba4-445a-b2df-5479cc9a2968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:31.810632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdecf3-4ba4-445a-b2df-5479cc9a2968 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:31.811014 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdecf3-4ba4-445a-b2df-5479cc9a2968 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:31.811432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdecf3-4ba4-445a-b2df-5479cc9a2968 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:31.812178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdecf3-4ba4-445a-b2df-5479cc9a2968 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:31.813686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdecf3-4ba4-445a-b2df-5479cc9a2968 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-4abdecf3-4ba4-445a-b2df-5479cc9a2968 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:31.825773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:31.826792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:31.828295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 342/696] 10.222.0.57 () {56 vars in 1129 bytes} [Tue Aug 9 18:30:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:31.837557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdecf3-4ba4-445a-b2df-5479cc9a2968 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:31.843770 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad64e0-9590-4127-b55c-dc5e207a01ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:31.844506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad64e0-9590-4127-b55c-dc5e207a01ea None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:31.844786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad64e0-9590-4127-b55c-dc5e207a01ea None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:31.845056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad64e0-9590-4127-b55c-dc5e207a01ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:31.852484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad64e0-9590-4127-b55c-dc5e207a01ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:31.854110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad64e0-9590-4127-b55c-dc5e207a01ea None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-5cad64e0-9590-4127-b55c-dc5e207a01ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:31.867820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:31.869846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:31.871255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:31.871255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:31.871255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:31.871255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:31.871255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:31.871255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:31.871255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:31.871255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:31.871255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 355/697] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:31.871255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad64e0-9590-4127-b55c-dc5e207a01ea None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:31.885350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:31.888236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:31.949917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdecf3-4ba4-445a-b2df-5479cc9a2968 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:31.989231 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad64e0-9590-4127-b55c-dc5e207a01ea None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:32.014847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5cc0e-5a7c-4a58-acf8-bc4ad6387137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:32.015505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5cc0e-5a7c-4a58-acf8-bc4ad6387137 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:32.015784 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5cc0e-5a7c-4a58-acf8-bc4ad6387137 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:32.016055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5cc0e-5a7c-4a58-acf8-bc4ad6387137 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:32.026311 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5cc0e-5a7c-4a58-acf8-bc4ad6387137 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:32.028193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5cc0e-5a7c-4a58-acf8-bc4ad6387137 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:32.037244 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bb564-57cf-4539-9bd9-2d224148a42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:32.038009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bb564-57cf-4539-9bd9-2d224148a42d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:32.038287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bb564-57cf-4539-9bd9-2d224148a42d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:32.038590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bb564-57cf-4539-9bd9-2d224148a42d None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:32.040689 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bb564-57cf-4539-9bd9-2d224148a42d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:32.040689 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bb564-57cf-4539-9bd9-2d224148a42d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-71b5cc0e-5a7c-4a58-acf8-bc4ad6387137 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:32.042380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:32.043287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:32.044193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:32.044193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:32.044193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:32.044193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:32.044193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:32.044193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:32.044193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:32.044193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:32.044193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:30:32.044193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:32.044723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bb564-57cf-4539-9bd9-2d224148a42d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:32.051079 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 356/698] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Aug 9 18:30:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:32.051596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 343/699] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 9 18:30:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:32.051834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bb564-57cf-4539-9bd9-2d224148a42d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:32.053251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5cc0e-5a7c-4a58-acf8-bc4ad6387137 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:32.143742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:32.170621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5cc0e-5a7c-4a58-acf8-bc4ad6387137 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:32.172369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:32.174941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bb564-57cf-4539-9bd9-2d224148a42d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:32.209075 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727873f-334b-455f-978e-3cf386d4a3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:32.210335 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727873f-334b-455f-978e-3cf386d4a3fb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:32.212427 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727873f-334b-455f-978e-3cf386d4a3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:32.213076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727873f-334b-455f-978e-3cf386d4a3fb None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:32.214102 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727873f-334b-455f-978e-3cf386d4a3fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:32.214762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727873f-334b-455f-978e-3cf386d4a3fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:32.216724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727873f-334b-455f-978e-3cf386d4a3fb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:32.217541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25195d7-61b9-454f-bc23-f84d2f79243d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:32.218181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25195d7-61b9-454f-bc23-f84d2f79243d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:32.218457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25195d7-61b9-454f-bc23-f84d2f79243d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:32.218726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25195d7-61b9-454f-bc23-f84d2f79243d None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:32.219430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25195d7-61b9-454f-bc23-f84d2f79243d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:32.220376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25195d7-61b9-454f-bc23-f84d2f79243d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:32.222130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25195d7-61b9-454f-bc23-f84d2f79243d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:32.235930 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 357/700] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:32.236876 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25195d7-61b9-454f-bc23-f84d2f79243d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:32.238700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 344/701] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 9 18:30:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:32.241380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727873f-334b-455f-978e-3cf386d4a3fb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:32.364111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727873f-334b-455f-978e-3cf386d4a3fb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:32.368562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25195d7-61b9-454f-bc23-f84d2f79243d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:32.399459 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:32.404523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce1d4d-4205-4ce7-ab13-4b16387f7a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:32.404951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e3a8a-4bf8-4aa0-856d-fe6bbf89f75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:32.405665 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e3a8a-4bf8-4aa0-856d-fe6bbf89f75e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:32.405848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e3a8a-4bf8-4aa0-856d-fe6bbf89f75e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:32.406121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e3a8a-4bf8-4aa0-856d-fe6bbf89f75e None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:32.406393 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce1d4d-4205-4ce7-ab13-4b16387f7a6d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:32.406864 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce1d4d-4205-4ce7-ab13-4b16387f7a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:32.407059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e3a8a-4bf8-4aa0-856d-fe6bbf89f75e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:32.407368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e3a8a-4bf8-4aa0-856d-fe6bbf89f75e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:32.407661 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce1d4d-4205-4ce7-ab13-4b16387f7a6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:32.408553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce1d4d-4205-4ce7-ab13-4b16387f7a6d None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=2ff735ecfcdf49089b5cfeafc56048cf) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:32.408748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e3a8a-4bf8-4aa0-856d-fe6bbf89f75e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:32.409270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce1d4d-4205-4ce7-ab13-4b16387f7a6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:32.410869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce1d4d-4205-4ce7-ab13-4b16387f7a6d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:32.415744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 358/702] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Aug 9 18:30:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:32.416609 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e3a8a-4bf8-4aa0-856d-fe6bbf89f75e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:32.419353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 345/703] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:32 2022] GET /identity/v3/role_assignments?user.id=2ff735ecfcdf49089b5cfeafc56048cf&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:32.572226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e3a8a-4bf8-4aa0-856d-fe6bbf89f75e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:32.639915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c78ce-29ba-4efc-a981-4e5c09621846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:32.640560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c78ce-29ba-4efc-a981-4e5c09621846 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:32.640796 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c78ce-29ba-4efc-a981-4e5c09621846 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:32.641843 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c78ce-29ba-4efc-a981-4e5c09621846 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:32.733684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:32.741408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c78ce-29ba-4efc-a981-4e5c09621846 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:32.742934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c78ce-29ba-4efc-a981-4e5c09621846 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:32.857072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 359/704] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:32.862400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce1d4d-4205-4ce7-ab13-4b16387f7a6d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:32.922922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:33.027190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce1d4d-4205-4ce7-ab13-4b16387f7a6d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:33.077545 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e46e69-7623-4859-a936-8643d4933165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:33.078448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e46e69-7623-4859-a936-8643d4933165 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:33.078877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e46e69-7623-4859-a936-8643d4933165 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:33.079313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e46e69-7623-4859-a936-8643d4933165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:33.091523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e46e69-7623-4859-a936-8643d4933165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:33.093195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e46e69-7623-4859-a936-8643d4933165 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:33.102097 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b41ce6-d463-495f-a6f2-bfddb1c3610c None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:33.102490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b41ce6-d463-495f-a6f2-bfddb1c3610c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:33.102872 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b41ce6-d463-495f-a6f2-bfddb1c3610c None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:33.104590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 360/705] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:33.109470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f3363-13f0-4329-ad91-c59ea7bfc75d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:33.109814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f3363-13f0-4329-ad91-c59ea7bfc75d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:33.110194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f3363-13f0-4329-ad91-c59ea7bfc75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-35e46e69-7623-4859-a936-8643d4933165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:33.111484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:33.112594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:33.113736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:33.113736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:33.113736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:33.115466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 346/706] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:33.122342 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e46e69-7623-4859-a936-8643d4933165 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:33.144045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-391f3363-13f0-4329-ad91-c59ea7bfc75d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:33.145389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-391f3363-13f0-4329-ad91-c59ea7bfc75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:33.225546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 361/707] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:33.235355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:33.276102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f3363-13f0-4329-ad91-c59ea7bfc75d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:33.302644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e46e69-7623-4859-a936-8643d4933165 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:33.323646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:33.395285 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e426d7c-e366-423d-8c29-91c7977b4d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:33.397055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e426d7c-e366-423d-8c29-91c7977b4d39 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:33.397511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e426d7c-e366-423d-8c29-91c7977b4d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:33.397999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e426d7c-e366-423d-8c29-91c7977b4d39 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:33.399006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e426d7c-e366-423d-8c29-91c7977b4d39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:33.399577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e426d7c-e366-423d-8c29-91c7977b4d39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:33.401208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e426d7c-e366-423d-8c29-91c7977b4d39 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:33.438814 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 347/708] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:33.440378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32995d-a45c-4b68-a90f-87669abd98bb None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:33.444228 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32995d-a45c-4b68-a90f-87669abd98bb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:33.445293 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32995d-a45c-4b68-a90f-87669abd98bb None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:33.447134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 348/709] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:33.448036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32995d-a45c-4b68-a90f-87669abd98bb None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:33.490398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f3363-13f0-4329-ad91-c59ea7bfc75d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:33.551707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db37a5-5e77-4842-837e-7e09e010f7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:33.553339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db37a5-5e77-4842-837e-7e09e010f7ad None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:33.554710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db37a5-5e77-4842-837e-7e09e010f7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:33.555203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db37a5-5e77-4842-837e-7e09e010f7ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:33.578162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db37a5-5e77-4842-837e-7e09e010f7ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:33.580755 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db37a5-5e77-4842-837e-7e09e010f7ad None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-67db37a5-5e77-4842-837e-7e09e010f7ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:33.612036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:33.613023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:33.613939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:33.613939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:33.613939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:33.621983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 362/710] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:33.624816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6e832-ffe6-410d-8874-40b5e29f6930 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:33.625096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6e832-ffe6-410d-8874-40b5e29f6930 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:33.625362 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6e832-ffe6-410d-8874-40b5e29f6930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:33.641512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32995d-a45c-4b68-a90f-87669abd98bb None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:33.652642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-afc6e832-ffe6-410d-8874-40b5e29f6930 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:33.656390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-afc6e832-ffe6-410d-8874-40b5e29f6930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:33.690403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d24318-4231-48be-b048-85c8bcf96f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:33.692065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d24318-4231-48be-b048-85c8bcf96f72 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:33.692869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d24318-4231-48be-b048-85c8bcf96f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:33.693666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d24318-4231-48be-b048-85c8bcf96f72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:33.698578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:33.704291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d24318-4231-48be-b048-85c8bcf96f72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:33.706053 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d24318-4231-48be-b048-85c8bcf96f72 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-d3d24318-4231-48be-b048-85c8bcf96f72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:33.727602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:33.728642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:33.729638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:33.729638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:33.729638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:33.729638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:33.729638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:33.729638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:33.729638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:33.729638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:33.732254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 349/711] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:33.733284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d24318-4231-48be-b048-85c8bcf96f72 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:33.776596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 363/712] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:33.786338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6e832-ffe6-410d-8874-40b5e29f6930 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:33.821596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:33.874706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d24318-4231-48be-b048-85c8bcf96f72 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:33.934739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6e832-ffe6-410d-8874-40b5e29f6930 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:33.948309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77b146-8703-4f42-92e1-8db4aed094fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:33.949442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77b146-8703-4f42-92e1-8db4aed094fb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:33.951881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77b146-8703-4f42-92e1-8db4aed094fb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:33.952544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77b146-8703-4f42-92e1-8db4aed094fb None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:33.953537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77b146-8703-4f42-92e1-8db4aed094fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:33.954071 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77b146-8703-4f42-92e1-8db4aed094fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:33.955855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77b146-8703-4f42-92e1-8db4aed094fb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:33.977817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08d234-c138-4bd7-a79c-f28c51413414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:33.978449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08d234-c138-4bd7-a79c-f28c51413414 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:33.978723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08d234-c138-4bd7-a79c-f28c51413414 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:33.978991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08d234-c138-4bd7-a79c-f28c51413414 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:33.979694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08d234-c138-4bd7-a79c-f28c51413414 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:33.979994 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08d234-c138-4bd7-a79c-f28c51413414 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:33.982750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08d234-c138-4bd7-a79c-f28c51413414 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:33.986625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 350/713] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:33.987846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77b146-8703-4f42-92e1-8db4aed094fb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:34.008256 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 364/714] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:34.009260 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08d234-c138-4bd7-a79c-f28c51413414 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:34.088216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:34.092323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:34.164842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08d234-c138-4bd7-a79c-f28c51413414 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:34.166258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77b146-8703-4f42-92e1-8db4aed094fb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:34.208416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19652ea6-8b5e-40d9-b3ee-bee4c8c5cf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:34.209112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19652ea6-8b5e-40d9-b3ee-bee4c8c5cf61 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:34.209511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19652ea6-8b5e-40d9-b3ee-bee4c8c5cf61 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:34.209904 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19652ea6-8b5e-40d9-b3ee-bee4c8c5cf61 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:34.211179 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19652ea6-8b5e-40d9-b3ee-bee4c8c5cf61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:34.213349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19652ea6-8b5e-40d9-b3ee-bee4c8c5cf61 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:34.228508 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606ac86-ecd4-4ec4-b6b3-417efd993843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:34.230193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606ac86-ecd4-4ec4-b6b3-417efd993843 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:34.230585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606ac86-ecd4-4ec4-b6b3-417efd993843 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:34.237703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606ac86-ecd4-4ec4-b6b3-417efd993843 None admin] PATH_INFO: `/v3/services/network` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:34.237703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606ac86-ecd4-4ec4-b6b3-417efd993843 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:34.237703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606ac86-ecd4-4ec4-b6b3-417efd993843 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-a606ac86-ecd4-4ec4-b6b3-417efd993843 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:34.242072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:34.243147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 18:30:34.244047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:34.248123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 351/715] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:33 2022] GET /identity/v3/services/network => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:34.248973 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606ac86-ecd4-4ec4-b6b3-417efd993843 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:34.289676 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 365/716] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:30:34 2022] POST /identity/v3/users => generated 254 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:34.290978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19652ea6-8b5e-40d9-b3ee-bee4c8c5cf61 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:34.533297 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19652ea6-8b5e-40d9-b3ee-bee4c8c5cf61 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:34.550712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606ac86-ecd4-4ec4-b6b3-417efd993843 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:34.564010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:34.584025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd0f41-84fd-4d58-b4d7-3bc9ced4a8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:34.584926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd0f41-84fd-4d58-b4d7-3bc9ced4a8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:34.585962 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd0f41-84fd-4d58-b4d7-3bc9ced4a8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:34.586379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd0f41-84fd-4d58-b4d7-3bc9ced4a8d4 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:34.587161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd0f41-84fd-4d58-b4d7-3bc9ced4a8d4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:34.589132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd0f41-84fd-4d58-b4d7-3bc9ced4a8d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:34.590685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd0f41-84fd-4d58-b4d7-3bc9ced4a8d4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:34.593052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:34.603226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 366/717] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:30:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:34.609995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd0f41-84fd-4d58-b4d7-3bc9ced4a8d4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:34.628865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed276ad9-e947-4209-9d54-65ee27534b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:34.629558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed276ad9-e947-4209-9d54-65ee27534b3b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:34.629887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed276ad9-e947-4209-9d54-65ee27534b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:34.630220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed276ad9-e947-4209-9d54-65ee27534b3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:34.631064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed276ad9-e947-4209-9d54-65ee27534b3b None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=e2ecd7d1cce644b0982cc23740537933) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:34.631389 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed276ad9-e947-4209-9d54-65ee27534b3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:34.633445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed276ad9-e947-4209-9d54-65ee27534b3b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:34.640110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 352/718] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:34 2022] GET /identity/v3/role_assignments?user.id=e2ecd7d1cce644b0982cc23740537933&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 259 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:34.743643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd0f41-84fd-4d58-b4d7-3bc9ced4a8d4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:34.786344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63962186-2e1f-4acd-b288-074b13fb3dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:34.787232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63962186-2e1f-4acd-b288-074b13fb3dfe None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:34.787703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63962186-2e1f-4acd-b288-074b13fb3dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:34.788166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63962186-2e1f-4acd-b288-074b13fb3dfe None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:34.789017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63962186-2e1f-4acd-b288-074b13fb3dfe None admin] RBAC: Adding query filter params (type=network) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:34.789491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63962186-2e1f-4acd-b288-074b13fb3dfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:34.791184 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63962186-2e1f-4acd-b288-074b13fb3dfe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:34.798277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 367/719] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:30:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:34.955507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa46dd8-df65-472e-bd4b-32bd499ae6c5 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:34.956026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa46dd8-df65-472e-bd4b-32bd499ae6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:34.956026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa46dd8-df65-472e-bd4b-32bd499ae6c5 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:34.959124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 353/720] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:34.964464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc2287-f462-480b-8a6b-073ab654ce59 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:34.964775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc2287-f462-480b-8a6b-073ab654ce59 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:34.965014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc2287-f462-480b-8a6b-073ab654ce59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:34.987776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1edc2287-f462-480b-8a6b-073ab654ce59 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:34.988208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1edc2287-f462-480b-8a6b-073ab654ce59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:35.060743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:35.081098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9bb0c-819a-4d4e-abb6-7eb51ffa3031 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:35.081619 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9bb0c-819a-4d4e-abb6-7eb51ffa3031 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:35.082102 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9bb0c-819a-4d4e-abb6-7eb51ffa3031 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:35.087955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 354/721] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:30:35.091737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 368/722] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:30:35.109161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec60378-7235-41e7-af8d-1934feaf3f94 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:35.109825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec60378-7235-41e7-af8d-1934feaf3f94 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:35.110307 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec60378-7235-41e7-af8d-1934feaf3f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:35.138240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9ec60378-7235-41e7-af8d-1934feaf3f94 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:35.139787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9ec60378-7235-41e7-af8d-1934feaf3f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:35.150478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc2287-f462-480b-8a6b-073ab654ce59 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:35.296395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 355/723] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:35.355970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec60378-7235-41e7-af8d-1934feaf3f94 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:35.388312 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc2287-f462-480b-8a6b-073ab654ce59 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:35.440925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2e1c0-ad92-435b-9bd2-f719aead2b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:35.441682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2e1c0-ad92-435b-9bd2-f719aead2b59 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:35.441963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2e1c0-ad92-435b-9bd2-f719aead2b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:35.442236 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2e1c0-ad92-435b-9bd2-f719aead2b59 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:35.445547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2e1c0-ad92-435b-9bd2-f719aead2b59 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:35.445547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2e1c0-ad92-435b-9bd2-f719aead2b59 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:35.535140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec60378-7235-41e7-af8d-1934feaf3f94 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:35.556568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:35.569255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 369/724] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:30:35 2022] POST /identity/v3/services => generated 251 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:35.623367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865c3b0-ffbb-4130-8580-62d500c2aff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:35.623939 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865c3b0-ffbb-4130-8580-62d500c2aff2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:35.624223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865c3b0-ffbb-4130-8580-62d500c2aff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:35.624502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865c3b0-ffbb-4130-8580-62d500c2aff2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:35.636734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865c3b0-ffbb-4130-8580-62d500c2aff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-9865c3b0-ffbb-4130-8580-62d500c2aff2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:35.642146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:35.643152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:35.643989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:35.643989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:35.643989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:35.643989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:35.643989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:35.643989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:35.643989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:35.643989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:35.643989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:35.643989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:35.643989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:35.643989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:35.654438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 356/725] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:35.654624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2e1c0-ad92-435b-9bd2-f719aead2b59 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:35.705433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:35.778774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2e1c0-ad92-435b-9bd2-f719aead2b59 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:35.818424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bde56-774a-494b-ab7a-d922e57f47d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:35.819071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bde56-774a-494b-ab7a-d922e57f47d4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:35.819354 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bde56-774a-494b-ab7a-d922e57f47d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:35.819764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bde56-774a-494b-ab7a-d922e57f47d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:35.821024 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bde56-774a-494b-ab7a-d922e57f47d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:35.821333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bde56-774a-494b-ab7a-d922e57f47d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:35.822826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bde56-774a-494b-ab7a-d922e57f47d4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:35.838559 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 370/726] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:35.840575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865c3b0-ffbb-4130-8580-62d500c2aff2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:35.978507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865c3b0-ffbb-4130-8580-62d500c2aff2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:36.021991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40200617-24f7-4706-bd07-e66af7db6ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:36.025558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40200617-24f7-4706-bd07-e66af7db6ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:36.025558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40200617-24f7-4706-bd07-e66af7db6ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:36.025558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40200617-24f7-4706-bd07-e66af7db6ee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:36.036078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40200617-24f7-4706-bd07-e66af7db6ee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:36.037662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40200617-24f7-4706-bd07-e66af7db6ee2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-40200617-24f7-4706-bd07-e66af7db6ee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:36.065111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:36.066147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:36.070648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:36.070648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:36.070648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:36.070648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 357/727] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:36.074511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bde56-774a-494b-ab7a-d922e57f47d4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:36.212425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bde56-774a-494b-ab7a-d922e57f47d4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:36.215538 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:36.273815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d3be4-21d8-4971-8af0-ad7d811efa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:36.273815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d3be4-21d8-4971-8af0-ad7d811efa7c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:36.273815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d3be4-21d8-4971-8af0-ad7d811efa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:36.273815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d3be4-21d8-4971-8af0-ad7d811efa7c None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:36.274213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d3be4-21d8-4971-8af0-ad7d811efa7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:36.274213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d3be4-21d8-4971-8af0-ad7d811efa7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:36.280185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d3be4-21d8-4971-8af0-ad7d811efa7c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:36.299949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 371/728] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:36.307775 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40200617-24f7-4706-bd07-e66af7db6ee2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:36.364298 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:36.540407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40200617-24f7-4706-bd07-e66af7db6ee2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:36.616318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096b5d8-550f-45c4-bfcc-65b52c462bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:36.616984 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096b5d8-550f-45c4-bfcc-65b52c462bdc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:36.617261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096b5d8-550f-45c4-bfcc-65b52c462bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:36.617553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096b5d8-550f-45c4-bfcc-65b52c462bdc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:36.635615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096b5d8-550f-45c4-bfcc-65b52c462bdc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:36.638206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096b5d8-550f-45c4-bfcc-65b52c462bdc None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-3096b5d8-550f-45c4-bfcc-65b52c462bdc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:36.671114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:36.672104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:36.673593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:36.673593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:36.673593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:36.673593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:36.673593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:36.673593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:36.673593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:36.673593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:36.673593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 18:30:36.673593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:36.673593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 358/729] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:36.678514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d3be4-21d8-4971-8af0-ad7d811efa7c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:36.951820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d3be4-21d8-4971-8af0-ad7d811efa7c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:37.024592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885bdd6-5f7e-43c8-b4cd-23df1efb92b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:37.025606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885bdd6-5f7e-43c8-b4cd-23df1efb92b2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:37.026081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885bdd6-5f7e-43c8-b4cd-23df1efb92b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:37.026546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885bdd6-5f7e-43c8-b4cd-23df1efb92b2 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:37.027837 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885bdd6-5f7e-43c8-b4cd-23df1efb92b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:37.028361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885bdd6-5f7e-43c8-b4cd-23df1efb92b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:37.031369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885bdd6-5f7e-43c8-b4cd-23df1efb92b2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:37.076371 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 372/730] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Aug 9 18:30:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:37.082603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096b5d8-550f-45c4-bfcc-65b52c462bdc None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:37.259139 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:37.284843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096b5d8-550f-45c4-bfcc-65b52c462bdc None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:37.349050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cd019-5665-4763-9ccb-908b1b8eef9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:37.349703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cd019-5665-4763-9ccb-908b1b8eef9d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:37.349987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cd019-5665-4763-9ccb-908b1b8eef9d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:37.350259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cd019-5665-4763-9ccb-908b1b8eef9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:37.455119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4120b-fbcc-4fb8-b957-bd89aadfc85c None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:37.455630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4120b-fbcc-4fb8-b957-bd89aadfc85c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:37.455789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4120b-fbcc-4fb8-b957-bd89aadfc85c None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:37.457043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 373/731] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:30:37.462789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5f44e-909b-4dcb-b5b8-c708cb5cf6a3 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:37.463691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5f44e-909b-4dcb-b5b8-c708cb5cf6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:37.464154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5f44e-909b-4dcb-b5b8-c708cb5cf6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:37.466898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cd019-5665-4763-9ccb-908b1b8eef9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:37.477546 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cd019-5665-4763-9ccb-908b1b8eef9d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:37.526749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-51a5f44e-909b-4dcb-b5b8-c708cb5cf6a3 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:37.530946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-51a5f44e-909b-4dcb-b5b8-c708cb5cf6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:37.575843 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:37.582350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:37.621668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 359/732] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:37 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 18:30:37.629693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cd019-5665-4763-9ccb-908b1b8eef9d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:37.730167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 374/733] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:37.732701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7a89d-aef8-4710-abad-b802745f97f7 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:37.733069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7a89d-aef8-4710-abad-b802745f97f7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:37.733612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7a89d-aef8-4710-abad-b802745f97f7 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:37.744912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 375/734] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:37 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:37.745891 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4d469-2a49-4cf0-97b4-9362cf2207e1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:37.746509 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4d469-2a49-4cf0-97b4-9362cf2207e1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:37.746992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4d469-2a49-4cf0-97b4-9362cf2207e1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:37.750070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 376/735] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:37.754328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affeb5f-e7d3-49a7-83af-afc3821bc02d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:37.754776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affeb5f-e7d3-49a7-83af-afc3821bc02d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:37.755247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affeb5f-e7d3-49a7-83af-afc3821bc02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:37.784707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6affeb5f-e7d3-49a7-83af-afc3821bc02d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:37.787098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6affeb5f-e7d3-49a7-83af-afc3821bc02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:37.791173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cd019-5665-4763-9ccb-908b1b8eef9d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:37.837623 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee70691-6b90-459a-af8f-fe1e9c050ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:37.838356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee70691-6b90-459a-af8f-fe1e9c050ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:37.838697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee70691-6b90-459a-af8f-fe1e9c050ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:37.839315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee70691-6b90-459a-af8f-fe1e9c050ca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:37.852113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee70691-6b90-459a-af8f-fe1e9c050ca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:37.853465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee70691-6b90-459a-af8f-fe1e9c050ca0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:37.874544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 377/736] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:37.876218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a16144-0ffb-4341-b768-9b591e07ceb0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:37.876724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a16144-0ffb-4341-b768-9b591e07ceb0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:37.877200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a16144-0ffb-4341-b768-9b591e07ceb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-fee70691-6b90-459a-af8f-fe1e9c050ca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:37.878061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:37.878980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:37.879861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:37.879861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:37.879861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:37.879861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 360/737] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:37.880500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee70691-6b90-459a-af8f-fe1e9c050ca0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:37.904744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d0a16144-0ffb-4341-b768-9b591e07ceb0 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:37.906225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d0a16144-0ffb-4341-b768-9b591e07ceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:37.928036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:37.990642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 378/738] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:30:37.991576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a16144-0ffb-4341-b768-9b591e07ceb0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:38.018259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:38.031494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee70691-6b90-459a-af8f-fe1e9c050ca0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:38.086027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282dd2b-5791-495d-aa3d-b22c3dda09b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:38.086741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282dd2b-5791-495d-aa3d-b22c3dda09b5 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:38.087123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282dd2b-5791-495d-aa3d-b22c3dda09b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:38.087470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282dd2b-5791-495d-aa3d-b22c3dda09b5 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:38.088145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282dd2b-5791-495d-aa3d-b22c3dda09b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:38.089669 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282dd2b-5791-495d-aa3d-b22c3dda09b5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:38.137834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a16144-0ffb-4341-b768-9b591e07ceb0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:38.177308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63edc3-df13-4af6-863c-9b92ace8f10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:38.178031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63edc3-df13-4af6-863c-9b92ace8f10a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:38.178316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63edc3-df13-4af6-863c-9b92ace8f10a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:38.178586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63edc3-df13-4af6-863c-9b92ace8f10a None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:38.179602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63edc3-df13-4af6-863c-9b92ace8f10a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:38.179902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63edc3-df13-4af6-863c-9b92ace8f10a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:38.181410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63edc3-df13-4af6-863c-9b92ace8f10a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:38.204676 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 379/739] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:38.204676 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63edc3-df13-4af6-863c-9b92ace8f10a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:38.262738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 361/740] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:30:37 2022] POST /identity/v3/services => generated 240 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:38.267580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282dd2b-5791-495d-aa3d-b22c3dda09b5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:38.317308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:38.393710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63edc3-df13-4af6-863c-9b92ace8f10a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:38.466961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:38.478401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fcadb-ca2e-4dd5-bf76-29ea7c2a898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:38.479067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fcadb-ca2e-4dd5-bf76-29ea7c2a898c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:38.479350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fcadb-ca2e-4dd5-bf76-29ea7c2a898c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:38.483012 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fcadb-ca2e-4dd5-bf76-29ea7c2a898c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:38.483135 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282dd2b-5791-495d-aa3d-b22c3dda09b5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:38.498757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fcadb-ca2e-4dd5-bf76-29ea7c2a898c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-847fcadb-ca2e-4dd5-bf76-29ea7c2a898c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:38.505199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:38.506340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:38.507933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:38.507933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:38.507933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:38.507933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:38.507933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:38.507933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:38.507933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:38.507933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:38.507933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:38.507933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:38.507933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:38.507933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:38.508721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 380/741] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:38.509791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fcadb-ca2e-4dd5-bf76-29ea7c2a898c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:38.564290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa94bdc-6f1a-4b4d-93ea-aa0575b4796a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:38.564290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa94bdc-6f1a-4b4d-93ea-aa0575b4796a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:38.564290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa94bdc-6f1a-4b4d-93ea-aa0575b4796a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:38.564290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa94bdc-6f1a-4b4d-93ea-aa0575b4796a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:38.594397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa94bdc-6f1a-4b4d-93ea-aa0575b4796a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-daa94bdc-6f1a-4b4d-93ea-aa0575b4796a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:38.602568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:38.603555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:38.604401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:38.604401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:38.604401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:38.604401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:38.604401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:38.604401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:38.604401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:38.604401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:38.604401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:38.604401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:38.604401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:38.604401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:38.605056 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 362/742] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:38.609269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa94bdc-6f1a-4b4d-93ea-aa0575b4796a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:38.669634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fcadb-ca2e-4dd5-bf76-29ea7c2a898c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:38.708225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:38.722295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b8e62-cd96-45c3-a7fa-46541a5c17a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:38.722945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b8e62-cd96-45c3-a7fa-46541a5c17a1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:38.725820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b8e62-cd96-45c3-a7fa-46541a5c17a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:38.725820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b8e62-cd96-45c3-a7fa-46541a5c17a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:38.736910 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b8e62-cd96-45c3-a7fa-46541a5c17a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:38.739108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b8e62-cd96-45c3-a7fa-46541a5c17a1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:38.747157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa94bdc-6f1a-4b4d-93ea-aa0575b4796a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-579b8e62-cd96-45c3-a7fa-46541a5c17a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:38.756364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:38.759786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:38.761867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:38.761867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:38.761867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:38.761867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:38.761867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:38.761867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:38.761867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:38.761867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:38.761867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 381/743] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:38.761867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b8e62-cd96-45c3-a7fa-46541a5c17a1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:38.801568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12043dda-5f99-468b-a23e-cb288311b852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:38.802242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12043dda-5f99-468b-a23e-cb288311b852 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:38.802688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12043dda-5f99-468b-a23e-cb288311b852 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:38.802960 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12043dda-5f99-468b-a23e-cb288311b852 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:38.804531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12043dda-5f99-468b-a23e-cb288311b852 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:38.804834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12043dda-5f99-468b-a23e-cb288311b852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:38.806902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12043dda-5f99-468b-a23e-cb288311b852 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:38.820898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 363/744] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:38.823144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccca44-ea13-4cad-9f53-29289019df17 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:38.823626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccca44-ea13-4cad-9f53-29289019df17 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:38.824304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccca44-ea13-4cad-9f53-29289019df17 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:38.826699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 364/745] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:38.828046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ccca44-ea13-4cad-9f53-29289019df17 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:38.858189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:38.900555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b8e62-cd96-45c3-a7fa-46541a5c17a1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:38.955266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcca02d-a4d6-4d01-b02a-774284f71eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:38.955913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcca02d-a4d6-4d01-b02a-774284f71eeb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:38.956260 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcca02d-a4d6-4d01-b02a-774284f71eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:38.956624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcca02d-a4d6-4d01-b02a-774284f71eeb None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:38.957290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcca02d-a4d6-4d01-b02a-774284f71eeb None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:38.957745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcca02d-a4d6-4d01-b02a-774284f71eeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:38.959174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcca02d-a4d6-4d01-b02a-774284f71eeb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:38.970355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 382/746] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:38.971269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcca02d-a4d6-4d01-b02a-774284f71eeb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:38.979464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ccca44-ea13-4cad-9f53-29289019df17 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:39.028932 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1206251-1add-44a0-8951-bf81a0c34fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:39.029956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1206251-1add-44a0-8951-bf81a0c34fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:39.030912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1206251-1add-44a0-8951-bf81a0c34fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:39.031367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1206251-1add-44a0-8951-bf81a0c34fe7 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:39.032665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1206251-1add-44a0-8951-bf81a0c34fe7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:39.033433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1206251-1add-44a0-8951-bf81a0c34fe7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:39.035615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1206251-1add-44a0-8951-bf81a0c34fe7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:39.041565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:39.059081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 365/747] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:39.059081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622b672-4d90-4928-9e62-83055f0cea46 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:39.059081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622b672-4d90-4928-9e62-83055f0cea46 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:39.059081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622b672-4d90-4928-9e62-83055f0cea46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:39.078861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a622b672-4d90-4928-9e62-83055f0cea46 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:39.080312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a622b672-4d90-4928-9e62-83055f0cea46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:39.109203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcca02d-a4d6-4d01-b02a-774284f71eeb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:39.162149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee906d7-382e-4803-a846-b3b48db40201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:39.162832 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee906d7-382e-4803-a846-b3b48db40201 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:39.163698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee906d7-382e-4803-a846-b3b48db40201 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:39.164021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee906d7-382e-4803-a846-b3b48db40201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:39.169675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 366/748] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:39.170614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622b672-4d90-4928-9e62-83055f0cea46 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:39.187028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee906d7-382e-4803-a846-b3b48db40201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:39.189582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee906d7-382e-4803-a846-b3b48db40201 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:39.195327 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-eee906d7-382e-4803-a846-b3b48db40201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:39.210909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:39.211973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:39.212854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:39.212854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:39.212854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:39.216903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 383/749] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:39.217953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee906d7-382e-4803-a846-b3b48db40201 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:39.333467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622b672-4d90-4928-9e62-83055f0cea46 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:39.357745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee906d7-382e-4803-a846-b3b48db40201 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:39.373760 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:39.376655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdcc40-22c1-436d-a026-08bb506bc914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:39.377021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdcc40-22c1-436d-a026-08bb506bc914 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:39.377364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdcc40-22c1-436d-a026-08bb506bc914 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:39.377676 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdcc40-22c1-436d-a026-08bb506bc914 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:39.389450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdcc40-22c1-436d-a026-08bb506bc914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:39.390940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdcc40-22c1-436d-a026-08bb506bc914 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:39.401167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec99ac6-5bea-4cf9-9a42-370a7f2f93cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:39.402241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec99ac6-5bea-4cf9-9a42-370a7f2f93cd None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:39.402526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec99ac6-5bea-4cf9-9a42-370a7f2f93cd None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:39.402801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec99ac6-5bea-4cf9-9a42-370a7f2f93cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:39.403544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec99ac6-5bea-4cf9-9a42-370a7f2f93cd None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=2ff735ecfcdf49089b5cfeafc56048cf) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:39.403983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec99ac6-5bea-4cf9-9a42-370a7f2f93cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:39.405397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec99ac6-5bea-4cf9-9a42-370a7f2f93cd None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-07cdcc40-22c1-436d-a026-08bb506bc914 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:39.411699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:39.412654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:39.414156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:39.414156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:39.414156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:39.422704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 384/750] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:39 2022] GET /identity/v3/role_assignments?user.id=2ff735ecfcdf49089b5cfeafc56048cf&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:39.422704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec99ac6-5bea-4cf9-9a42-370a7f2f93cd None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:39.422990 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 367/751] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:39.422990 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdcc40-22c1-436d-a026-08bb506bc914 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:39.540517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:39.587499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec99ac6-5bea-4cf9-9a42-370a7f2f93cd None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:39.629263 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdcc40-22c1-436d-a026-08bb506bc914 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:39.669664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb31b8e-1de9-44ee-a3a7-a010962eb20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:39.670275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb31b8e-1de9-44ee-a3a7-a010962eb20b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:39.670558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb31b8e-1de9-44ee-a3a7-a010962eb20b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:39.670831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb31b8e-1de9-44ee-a3a7-a010962eb20b None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:39.671298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b215372-bf9a-46af-b359-0ce573694fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:39.671470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb31b8e-1de9-44ee-a3a7-a010962eb20b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:39.671778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb31b8e-1de9-44ee-a3a7-a010962eb20b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:39.672562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b215372-bf9a-46af-b359-0ce573694fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:39.672846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b215372-bf9a-46af-b359-0ce573694fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:39.673117 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b215372-bf9a-46af-b359-0ce573694fa1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:39.673218 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb31b8e-1de9-44ee-a3a7-a010962eb20b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:39.676135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b215372-bf9a-46af-b359-0ce573694fa1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:39.676135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b215372-bf9a-46af-b359-0ce573694fa1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-4b215372-bf9a-46af-b359-0ce573694fa1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:39.699473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:39.700506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 18:30:39.701344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:39.702999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 368/753] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:39.704243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb31b8e-1de9-44ee-a3a7-a010962eb20b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:39.704668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 385/753] 10.222.0.57 () {56 vars in 1129 bytes} [Tue Aug 9 18:30:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:39.706072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b215372-bf9a-46af-b359-0ce573694fa1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:39.752011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:39.851471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b215372-bf9a-46af-b359-0ce573694fa1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:39.886223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb31b8e-1de9-44ee-a3a7-a010962eb20b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:39.901666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d3013-c516-4699-a71e-28b05f438925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:39.902329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d3013-c516-4699-a71e-28b05f438925 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:39.902870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d3013-c516-4699-a71e-28b05f438925 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:39.903244 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d3013-c516-4699-a71e-28b05f438925 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:39.914320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d3013-c516-4699-a71e-28b05f438925 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:39.914320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d3013-c516-4699-a71e-28b05f438925 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-622d3013-c516-4699-a71e-28b05f438925 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:39.928070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:39.929556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:39.930706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:39.930706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:39.930706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:39.930706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:39.930706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:39.930706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:39.930706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:39.930706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:39.930706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:30:39.930706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:39.934537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 386/754] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 9 18:30:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:39.934746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d3013-c516-4699-a71e-28b05f438925 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:39.953812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:40.006272 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524403a-f548-49ef-b7a9-95070134e834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:40.007498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524403a-f548-49ef-b7a9-95070134e834 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:40.007824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524403a-f548-49ef-b7a9-95070134e834 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:40.008095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524403a-f548-49ef-b7a9-95070134e834 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:40.008758 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524403a-f548-49ef-b7a9-95070134e834 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:40.009063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524403a-f548-49ef-b7a9-95070134e834 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:40.010575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524403a-f548-49ef-b7a9-95070134e834 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:40.037658 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 369/755] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:40.037658 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524403a-f548-49ef-b7a9-95070134e834 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:40.084506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:40.096541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d3013-c516-4699-a71e-28b05f438925 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:40.160059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c1f66-bc26-489c-9606-e72502afa2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:40.160617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c1f66-bc26-489c-9606-e72502afa2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:40.160898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c1f66-bc26-489c-9606-e72502afa2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:40.161165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c1f66-bc26-489c-9606-e72502afa2f4 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:40.161834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c1f66-bc26-489c-9606-e72502afa2f4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:40.162140 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c1f66-bc26-489c-9606-e72502afa2f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:40.163556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c1f66-bc26-489c-9606-e72502afa2f4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:40.168979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 387/756] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Aug 9 18:30:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:40.170082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c1f66-bc26-489c-9606-e72502afa2f4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:40.235704 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524403a-f548-49ef-b7a9-95070134e834 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:40.294801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c1f66-bc26-489c-9606-e72502afa2f4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:40.334671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43830bed-d68a-44f6-98ce-f490748c983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:40.335666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43830bed-d68a-44f6-98ce-f490748c983d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:40.335886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43830bed-d68a-44f6-98ce-f490748c983d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:40.336183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43830bed-d68a-44f6-98ce-f490748c983d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:40.345761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43830bed-d68a-44f6-98ce-f490748c983d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:40.347192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43830bed-d68a-44f6-98ce-f490748c983d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:40.355062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae17ab-0b99-4afb-8e0f-b4e8d37aa801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:40.355685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae17ab-0b99-4afb-8e0f-b4e8d37aa801 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:40.355967 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae17ab-0b99-4afb-8e0f-b4e8d37aa801 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:40.356243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae17ab-0b99-4afb-8e0f-b4e8d37aa801 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:40.356991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae17ab-0b99-4afb-8e0f-b4e8d37aa801 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:40.357298 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae17ab-0b99-4afb-8e0f-b4e8d37aa801 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-43830bed-d68a-44f6-98ce-f490748c983d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:40.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:40.361803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:40.362720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:40.362720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:40.362720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:40.362720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:40.362720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:40.362720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:40.362720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:40.362720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:40.362720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 18:30:40.362720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:40.362720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 388/757] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 9 18:30:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:40.363287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43830bed-d68a-44f6-98ce-f490748c983d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:40.368870 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae17ab-0b99-4afb-8e0f-b4e8d37aa801 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:40.389347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:40.405629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 370/758] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 9 18:30:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:40.407563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae17ab-0b99-4afb-8e0f-b4e8d37aa801 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:40.480991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:40.499106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43830bed-d68a-44f6-98ce-f490748c983d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:40.547838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd8083-e3b6-4e64-81f9-1d3225e9c93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:40.548447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd8083-e3b6-4e64-81f9-1d3225e9c93e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:40.548841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd8083-e3b6-4e64-81f9-1d3225e9c93e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:40.549113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd8083-e3b6-4e64-81f9-1d3225e9c93e None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:40.549814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd8083-e3b6-4e64-81f9-1d3225e9c93e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:40.550540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd8083-e3b6-4e64-81f9-1d3225e9c93e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:40.552281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd8083-e3b6-4e64-81f9-1d3225e9c93e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:40.558208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 389/759] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Aug 9 18:30:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:40.559285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd8083-e3b6-4e64-81f9-1d3225e9c93e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:40.573980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae17ab-0b99-4afb-8e0f-b4e8d37aa801 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:40.631806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5e9b4-e7fe-4fc3-a6c0-740e1abe39d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:40.632445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5e9b4-e7fe-4fc3-a6c0-740e1abe39d4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:40.632725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5e9b4-e7fe-4fc3-a6c0-740e1abe39d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:40.632995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5e9b4-e7fe-4fc3-a6c0-740e1abe39d4 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:40.633811 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5e9b4-e7fe-4fc3-a6c0-740e1abe39d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:40.634122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5e9b4-e7fe-4fc3-a6c0-740e1abe39d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:40.635611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5e9b4-e7fe-4fc3-a6c0-740e1abe39d4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:40.673582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 371/760] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Aug 9 18:30:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:40.673582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f846e8-b022-4b51-bb42-383414f4fd13 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:40.673582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f846e8-b022-4b51-bb42-383414f4fd13 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:40.674496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f846e8-b022-4b51-bb42-383414f4fd13 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:40.676089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 372/761] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:40.676751 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:40.678498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f846e8-b022-4b51-bb42-383414f4fd13 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:40.712663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd8083-e3b6-4e64-81f9-1d3225e9c93e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:40.762606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc6aeb-1fcb-4ded-b553-d519b911a040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:40.763800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc6aeb-1fcb-4ded-b553-d519b911a040 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:40.764092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc6aeb-1fcb-4ded-b553-d519b911a040 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:40.764372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc6aeb-1fcb-4ded-b553-d519b911a040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:40.782042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc6aeb-1fcb-4ded-b553-d519b911a040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:40.784352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc6aeb-1fcb-4ded-b553-d519b911a040 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-3edc6aeb-1fcb-4ded-b553-d519b911a040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:40.802959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:40.804590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:40.805495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:40.805495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:40.805495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:40.805708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:40.805786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 390/762] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:40.806083 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc6aeb-1fcb-4ded-b553-d519b911a040 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:40.814965 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f846e8-b022-4b51-bb42-383414f4fd13 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:40.868148 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621aca9-2e76-4e8c-9430-c84c7d474b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:40.868764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621aca9-2e76-4e8c-9430-c84c7d474b40 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:40.869045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621aca9-2e76-4e8c-9430-c84c7d474b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:40.869317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621aca9-2e76-4e8c-9430-c84c7d474b40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:40.870346 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621aca9-2e76-4e8c-9430-c84c7d474b40 None admin] RBAC: Adding query filter params (interface=public, service_id=a97dc3667c4e4bafae997d4028b2f6e1, region_id=RegionOne) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:40.870608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621aca9-2e76-4e8c-9430-c84c7d474b40 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:40.872120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621aca9-2e76-4e8c-9430-c84c7d474b40 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:40.887398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 373/763] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 9 18:30:40 2022] GET /identity/v3/endpoints?service_id=a97dc3667c4e4bafae997d4028b2f6e1&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:40.888859 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ada45f-bd7e-4c50-8e1e-9269828a0ed6 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:40.889147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ada45f-bd7e-4c50-8e1e-9269828a0ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:40.889457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ada45f-bd7e-4c50-8e1e-9269828a0ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:40.909482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-34ada45f-bd7e-4c50-8e1e-9269828a0ed6 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:40.910706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-34ada45f-bd7e-4c50-8e1e-9269828a0ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:40.954718 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc6aeb-1fcb-4ded-b553-d519b911a040 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:40.989459 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:40.999662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234bb0d-3ce9-4eb0-8ea7-82e363731ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:41.013711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234bb0d-3ce9-4eb0-8ea7-82e363731ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:41.013711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234bb0d-3ce9-4eb0-8ea7-82e363731ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:41.013711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234bb0d-3ce9-4eb0-8ea7-82e363731ef0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:41.014054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 374/764] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:41.014054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ada45f-bd7e-4c50-8e1e-9269828a0ed6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:41.123265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:41.145556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ada45f-bd7e-4c50-8e1e-9269828a0ed6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:41.147060 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234bb0d-3ce9-4eb0-8ea7-82e363731ef0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:41.147946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234bb0d-3ce9-4eb0-8ea7-82e363731ef0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:41.191862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6cf79-0d32-4a15-aead-cc8f0640baa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:41.192508 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6cf79-0d32-4a15-aead-cc8f0640baa1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:41.192859 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6cf79-0d32-4a15-aead-cc8f0640baa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:41.193140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6cf79-0d32-4a15-aead-cc8f0640baa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:41.193941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6cf79-0d32-4a15-aead-cc8f0640baa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:41.194252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6cf79-0d32-4a15-aead-cc8f0640baa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:41.196581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6cf79-0d32-4a15-aead-cc8f0640baa1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:41.209178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 375/765] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:41.210130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6cf79-0d32-4a15-aead-cc8f0640baa1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:41.292943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 391/766] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:30:41.293931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234bb0d-3ce9-4eb0-8ea7-82e363731ef0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:41.335088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:41.376792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6cf79-0d32-4a15-aead-cc8f0640baa1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:41.417776 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ab251-45d7-4f11-9ac2-6adc9e73c128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:41.418570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ab251-45d7-4f11-9ac2-6adc9e73c128 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:41.418920 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ab251-45d7-4f11-9ac2-6adc9e73c128 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:41.419263 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ab251-45d7-4f11-9ac2-6adc9e73c128 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:41.420115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ab251-45d7-4f11-9ac2-6adc9e73c128 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:41.421999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ab251-45d7-4f11-9ac2-6adc9e73c128 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:41.435099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 376/767] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 9 18:30:41 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:41.435294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234bb0d-3ce9-4eb0-8ea7-82e363731ef0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:41.435376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ab251-45d7-4f11-9ac2-6adc9e73c128 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:41.457579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:41.504768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03706c0-f5c6-4f8e-bf88-7dbad90cf884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:41.506061 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03706c0-f5c6-4f8e-bf88-7dbad90cf884 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:41.506624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03706c0-f5c6-4f8e-bf88-7dbad90cf884 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:41.507510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03706c0-f5c6-4f8e-bf88-7dbad90cf884 None admin] PATH_INFO: `/v3/services/network` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:41.509494 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03706c0-f5c6-4f8e-bf88-7dbad90cf884 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:41.515388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03706c0-f5c6-4f8e-bf88-7dbad90cf884 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-a03706c0-f5c6-4f8e-bf88-7dbad90cf884 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:41.533206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:41.539118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:41.539998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 392/768] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:41 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:41.540874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03706c0-f5c6-4f8e-bf88-7dbad90cf884 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:41.636467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ab251-45d7-4f11-9ac2-6adc9e73c128 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:41.688104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892a122-f085-4903-a3ca-f40ddc04d548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:41.689578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892a122-f085-4903-a3ca-f40ddc04d548 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:41.689578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892a122-f085-4903-a3ca-f40ddc04d548 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:41.689578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892a122-f085-4903-a3ca-f40ddc04d548 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:41.701026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:41.704533 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892a122-f085-4903-a3ca-f40ddc04d548 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:41.708616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892a122-f085-4903-a3ca-f40ddc04d548 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:41.713292 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03706c0-f5c6-4f8e-bf88-7dbad90cf884 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-c892a122-f085-4903-a3ca-f40ddc04d548 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:41.727150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:41.728205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:41.729316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:41.729316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:41.729316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:41.729316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:41.729316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:41.729316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:41.729316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:41.729316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:41.736092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 377/769] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:41.736092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892a122-f085-4903-a3ca-f40ddc04d548 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:41.767050 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f92da-3869-4d13-bcdf-6af4af022ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:41.768363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f92da-3869-4d13-bcdf-6af4af022ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:41.768827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f92da-3869-4d13-bcdf-6af4af022ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:41.769296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f92da-3869-4d13-bcdf-6af4af022ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:41.784305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f92da-3869-4d13-bcdf-6af4af022ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:41.785750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f92da-3869-4d13-bcdf-6af4af022ea3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-0e1f92da-3869-4d13-bcdf-6af4af022ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:41.803207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:41.804521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:41.805380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:41.805380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:41.805380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:41.809816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 393/770] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:41.811175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90642948-859b-47aa-8301-9b23a6c00909 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:41.811795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90642948-859b-47aa-8301-9b23a6c00909 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:41.811795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90642948-859b-47aa-8301-9b23a6c00909 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:41.813028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 394/771] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:41.813970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90642948-859b-47aa-8301-9b23a6c00909 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:41.829275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:41.866306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892a122-f085-4903-a3ca-f40ddc04d548 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:41.916764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf70b6-2d96-40a8-9817-c1c4aa73b95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:41.917649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf70b6-2d96-40a8-9817-c1c4aa73b95b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:41.917933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf70b6-2d96-40a8-9817-c1c4aa73b95b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:41.918207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf70b6-2d96-40a8-9817-c1c4aa73b95b None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:41.918844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf70b6-2d96-40a8-9817-c1c4aa73b95b None admin] RBAC: Adding query filter params (name=network) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:41.919151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf70b6-2d96-40a8-9817-c1c4aa73b95b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:41.921028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf70b6-2d96-40a8-9817-c1c4aa73b95b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:41.927348 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 378/772] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:30:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:41.928790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf70b6-2d96-40a8-9817-c1c4aa73b95b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:41.946299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90642948-859b-47aa-8301-9b23a6c00909 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:42.003544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6e840-8c90-4aef-95c4-c6322e4a5770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:42.003544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6e840-8c90-4aef-95c4-c6322e4a5770 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:42.003544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6e840-8c90-4aef-95c4-c6322e4a5770 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:42.003544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6e840-8c90-4aef-95c4-c6322e4a5770 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:42.003991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6e840-8c90-4aef-95c4-c6322e4a5770 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:42.003991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6e840-8c90-4aef-95c4-c6322e4a5770 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:42.005013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6e840-8c90-4aef-95c4-c6322e4a5770 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:42.028774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 395/773] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:42.028774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d3f5e-8b1e-41ea-9c83-9e8080f2261b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:42.028774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d3f5e-8b1e-41ea-9c83-9e8080f2261b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:42.028774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d3f5e-8b1e-41ea-9c83-9e8080f2261b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:42.046015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:42.048776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0d4d3f5e-8b1e-41ea-9c83-9e8080f2261b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:42.050019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0d4d3f5e-8b1e-41ea-9c83-9e8080f2261b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:42.069928 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf70b6-2d96-40a8-9817-c1c4aa73b95b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:42.121598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce9d67-b2da-4f91-b392-efc229e9d6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:42.122366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce9d67-b2da-4f91-b392-efc229e9d6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:42.122755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce9d67-b2da-4f91-b392-efc229e9d6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:42.123119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce9d67-b2da-4f91-b392-efc229e9d6e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:42.123858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce9d67-b2da-4f91-b392-efc229e9d6e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:42.124691 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce9d67-b2da-4f91-b392-efc229e9d6e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:42.125738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce9d67-b2da-4f91-b392-efc229e9d6e4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:42.129197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:42.146287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 379/774] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:42.147454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce9d67-b2da-4f91-b392-efc229e9d6e4 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:42.151394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 396/775] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:42.152780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d3f5e-8b1e-41ea-9c83-9e8080f2261b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:42.270977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce9d67-b2da-4f91-b392-efc229e9d6e4 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:42.308424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d3f5e-8b1e-41ea-9c83-9e8080f2261b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:42.317548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f44ff-aad9-43a7-a639-ed6fbda5bb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:42.317548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f44ff-aad9-43a7-a639-ed6fbda5bb04 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:42.317548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f44ff-aad9-43a7-a639-ed6fbda5bb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:42.317548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f44ff-aad9-43a7-a639-ed6fbda5bb04 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:42.317959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f44ff-aad9-43a7-a639-ed6fbda5bb04 None admin] RBAC: Adding query filter params (type=network) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:42.317959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f44ff-aad9-43a7-a639-ed6fbda5bb04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:42.319086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f44ff-aad9-43a7-a639-ed6fbda5bb04 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:42.329163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 380/776] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:30:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:42.330429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f44ff-aad9-43a7-a639-ed6fbda5bb04 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:42.366836 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f1228-a008-4f69-ac9f-1559d294b971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:42.367528 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f1228-a008-4f69-ac9f-1559d294b971 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:42.367810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f1228-a008-4f69-ac9f-1559d294b971 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:42.368086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f1228-a008-4f69-ac9f-1559d294b971 None admin] PATH_INFO: `/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/e2ecd7d1cce644b0982cc23740537933/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:42.370098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f1228-a008-4f69-ac9f-1559d294b971 None admin] RBAC: Authorizing `identity:create_grant(project_id=abcccf694e7b4f66a7372e0ede416caf, user_id=e2ecd7d1cce644b0982cc23740537933, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:42.371638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f1228-a008-4f69-ac9f-1559d294b971 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:42.375051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:42.407006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:42.449472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 397/777] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:30:42 2022] PUT /identity/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/e2ecd7d1cce644b0982cc23740537933/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:30:42.450205 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f1228-a008-4f69-ac9f-1559d294b971 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:42.474011 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f44ff-aad9-43a7-a639-ed6fbda5bb04 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:42.534542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb830c60-ff57-4e83-96a8-f83a49b7a136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:42.535712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb830c60-ff57-4e83-96a8-f83a49b7a136 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:42.536961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb830c60-ff57-4e83-96a8-f83a49b7a136 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:42.537581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb830c60-ff57-4e83-96a8-f83a49b7a136 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:42.547201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb830c60-ff57-4e83-96a8-f83a49b7a136 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:42.549354 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb830c60-ff57-4e83-96a8-f83a49b7a136 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-eb830c60-ff57-4e83-96a8-f83a49b7a136 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:42.565946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:42.566868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:42.567748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:42.567748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:42.567748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:42.567748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:42.567748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:42.567748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:42.567748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:42.567748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:42.573278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 381/778] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:42.574521 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb830c60-ff57-4e83-96a8-f83a49b7a136 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:42.651374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f1228-a008-4f69-ac9f-1559d294b971 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:42.715363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb830c60-ff57-4e83-96a8-f83a49b7a136 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:42.719781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de966d87-25ff-4667-a9ef-258b8f11dacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:42.720499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de966d87-25ff-4667-a9ef-258b8f11dacb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:42.720784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de966d87-25ff-4667-a9ef-258b8f11dacb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:42.721053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de966d87-25ff-4667-a9ef-258b8f11dacb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:42.740833 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:42.746829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de966d87-25ff-4667-a9ef-258b8f11dacb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-de966d87-25ff-4667-a9ef-258b8f11dacb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:42.755302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:42.758838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:42.759703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:42.759703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:42.759703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:42.759703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:42.759703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:42.759703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:42.759703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:42.759703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:42.759703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:42.759703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:42.759703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:42.759703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:42.760763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0a9fc-9e34-4b40-928f-037718edc1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:42.761076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0a9fc-9e34-4b40-928f-037718edc1ff None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:42.761365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0a9fc-9e34-4b40-928f-037718edc1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:42.761697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0a9fc-9e34-4b40-928f-037718edc1ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:42.762618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0a9fc-9e34-4b40-928f-037718edc1ff None admin] RBAC: Adding query filter params (interface=public, service_id=35ff358e5c1045ce953da678a1bdf33a, region_id=RegionOne) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:42.762726 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0a9fc-9e34-4b40-928f-037718edc1ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:42.763144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 398/779] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:42.764072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0a9fc-9e34-4b40-928f-037718edc1ff None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:42.764408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de966d87-25ff-4667-a9ef-258b8f11dacb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:42.780218 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 382/780] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 9 18:30:42 2022] GET /identity/v3/endpoints?service_id=35ff358e5c1045ce953da678a1bdf33a&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:42.781433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0a9fc-9e34-4b40-928f-037718edc1ff None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:42.889658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:42.954441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de966d87-25ff-4667-a9ef-258b8f11dacb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:42.960719 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0a9fc-9e34-4b40-928f-037718edc1ff None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:43.003026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918b62f-e870-42c0-8127-06cf43656ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:43.004787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918b62f-e870-42c0-8127-06cf43656ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:43.005252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918b62f-e870-42c0-8127-06cf43656ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:43.005854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918b62f-e870-42c0-8127-06cf43656ab8 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:43.007334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918b62f-e870-42c0-8127-06cf43656ab8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:43.007856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918b62f-e870-42c0-8127-06cf43656ab8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:43.009586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918b62f-e870-42c0-8127-06cf43656ab8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:43.020294 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b3d64-1268-4363-911e-b4f48fed22c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:43.020923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b3d64-1268-4363-911e-b4f48fed22c4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:43.021226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b3d64-1268-4363-911e-b4f48fed22c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:43.021476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b3d64-1268-4363-911e-b4f48fed22c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:43.022232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b3d64-1268-4363-911e-b4f48fed22c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:43.023121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b3d64-1268-4363-911e-b4f48fed22c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:43.024727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b3d64-1268-4363-911e-b4f48fed22c4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:43.032571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 399/781] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:43.033818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918b62f-e870-42c0-8127-06cf43656ab8 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:43.053162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 383/782] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:43.055495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b3d64-1268-4363-911e-b4f48fed22c4 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:43.166015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:43.224748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918b62f-e870-42c0-8127-06cf43656ab8 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:43.251692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:43.279268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2091a2b-9eeb-4552-a9d3-531cf0fdfc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:43.280250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2091a2b-9eeb-4552-a9d3-531cf0fdfc8e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:43.280782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2091a2b-9eeb-4552-a9d3-531cf0fdfc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:43.281259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2091a2b-9eeb-4552-a9d3-531cf0fdfc8e None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:43.282179 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2091a2b-9eeb-4552-a9d3-531cf0fdfc8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:43.284230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2091a2b-9eeb-4552-a9d3-531cf0fdfc8e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:43.297809 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b3d64-1268-4363-911e-b4f48fed22c4 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:43.298534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 400/783] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 9 18:30:43 2022] GET /identity/v3/services => generated 772 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:43.300774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2091a2b-9eeb-4552-a9d3-531cf0fdfc8e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:43.383176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00052ffe-5ff8-4e33-8903-8892540fc2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:43.383805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00052ffe-5ff8-4e33-8903-8892540fc2ba None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:43.384142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00052ffe-5ff8-4e33-8903-8892540fc2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:43.384418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00052ffe-5ff8-4e33-8903-8892540fc2ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:43.385553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00052ffe-5ff8-4e33-8903-8892540fc2ba None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=1255cb6e360244ba9d5cec5cc65a1f8a) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:43.385553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00052ffe-5ff8-4e33-8903-8892540fc2ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:43.386972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00052ffe-5ff8-4e33-8903-8892540fc2ba None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:43.399683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 384/784] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:43 2022] GET /identity/v3/role_assignments?user.id=1255cb6e360244ba9d5cec5cc65a1f8a&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 259 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:43.537131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2091a2b-9eeb-4552-a9d3-531cf0fdfc8e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:43.593914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f64129-a0fa-4e6e-875f-7cce52f99c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:43.594261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f64129-a0fa-4e6e-875f-7cce52f99c43 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:43.594614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f64129-a0fa-4e6e-875f-7cce52f99c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:43.594738 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f64129-a0fa-4e6e-875f-7cce52f99c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:43.615118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f64129-a0fa-4e6e-875f-7cce52f99c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:43.616689 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f64129-a0fa-4e6e-875f-7cce52f99c43 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-46f64129-a0fa-4e6e-875f-7cce52f99c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:43.634561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:43.635580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:43.636455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:43.636455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:43.636455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:43.636455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 401/785] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:43.642009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00052ffe-5ff8-4e33-8903-8892540fc2ba None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:43.809854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042a7f7-d65d-470b-ac9e-3f39d1ed4274 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:43.810211 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042a7f7-d65d-470b-ac9e-3f39d1ed4274 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:43.810427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042a7f7-d65d-470b-ac9e-3f39d1ed4274 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:43.818003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 402/786] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:43.823530 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05faa9-b787-4370-82ef-5deeb3075239 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:43.823817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05faa9-b787-4370-82ef-5deeb3075239 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:43.824094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05faa9-b787-4370-82ef-5deeb3075239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:43.855110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00052ffe-5ff8-4e33-8903-8892540fc2ba None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:43.868673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ce05faa9-b787-4370-82ef-5deeb3075239 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:43.869927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ce05faa9-b787-4370-82ef-5deeb3075239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:43.886367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:43.929918 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:43.939254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a991c2c-983e-47a4-8d2e-14c326a108ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:43.940717 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a991c2c-983e-47a4-8d2e-14c326a108ae None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:43.942370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a991c2c-983e-47a4-8d2e-14c326a108ae None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:43.942884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a991c2c-983e-47a4-8d2e-14c326a108ae None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:43.946375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a991c2c-983e-47a4-8d2e-14c326a108ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:43.947405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a991c2c-983e-47a4-8d2e-14c326a108ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:43.951253 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a991c2c-983e-47a4-8d2e-14c326a108ae None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:43.983795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 385/787] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:43.987277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a991c2c-983e-47a4-8d2e-14c326a108ae None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:44.034605 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 403/788] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:44.067936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05faa9-b787-4370-82ef-5deeb3075239 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:44.243621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a991c2c-983e-47a4-8d2e-14c326a108ae None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:44.309997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05faa9-b787-4370-82ef-5deeb3075239 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:44.330446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4849bde-6924-403c-8ba2-8dc251343a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:44.331482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4849bde-6924-403c-8ba2-8dc251343a5c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:44.332035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4849bde-6924-403c-8ba2-8dc251343a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:44.332563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4849bde-6924-403c-8ba2-8dc251343a5c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:44.347047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4849bde-6924-403c-8ba2-8dc251343a5c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:44.353745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4849bde-6924-403c-8ba2-8dc251343a5c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:44.368373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb9bb5-4302-4f28-b858-1c704acb0393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:44.369021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb9bb5-4302-4f28-b858-1c704acb0393 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:44.369296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb9bb5-4302-4f28-b858-1c704acb0393 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:44.369583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb9bb5-4302-4f28-b858-1c704acb0393 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:44.370175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb9bb5-4302-4f28-b858-1c704acb0393 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:44.371547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb9bb5-4302-4f28-b858-1c704acb0393 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-e4849bde-6924-403c-8ba2-8dc251343a5c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:44.389284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:44.390375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:44.391336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:44.391336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:44.391336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:44.391336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:44.391336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:44.391336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:44.391336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:44.391336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:44.391336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 18:30:44.391336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:44.396345 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 386/789] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:44.402291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4849bde-6924-403c-8ba2-8dc251343a5c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-c6cb9bb5-4302-4f28-b858-1c704acb0393 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:44.415239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:44.416273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:44.417138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 404/790] 10.222.0.57 () {56 vars in 1129 bytes} [Tue Aug 9 18:30:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:44.425602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb9bb5-4302-4f28-b858-1c704acb0393 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:44.468602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:44.479518 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:44.616283 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4849bde-6924-403c-8ba2-8dc251343a5c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:44.632688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb9bb5-4302-4f28-b858-1c704acb0393 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:44.694072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe13cc-dff7-4f2d-8469-132f68f96378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:44.694799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe13cc-dff7-4f2d-8469-132f68f96378 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:44.694969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe13cc-dff7-4f2d-8469-132f68f96378 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:44.696442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe13cc-dff7-4f2d-8469-132f68f96378 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:44.696442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe13cc-dff7-4f2d-8469-132f68f96378 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:44.696442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe13cc-dff7-4f2d-8469-132f68f96378 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:44.697683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe13cc-dff7-4f2d-8469-132f68f96378 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:44.713548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cb8d4-9a6a-4ef7-bbe1-e1a376a171ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:44.713548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cb8d4-9a6a-4ef7-bbe1-e1a376a171ef None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:44.713548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cb8d4-9a6a-4ef7-bbe1-e1a376a171ef None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:44.713548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cb8d4-9a6a-4ef7-bbe1-e1a376a171ef None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:44.713984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cb8d4-9a6a-4ef7-bbe1-e1a376a171ef None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:44.713984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cb8d4-9a6a-4ef7-bbe1-e1a376a171ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:44.715646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cb8d4-9a6a-4ef7-bbe1-e1a376a171ef None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:44.734335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 405/791] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Aug 9 18:30:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:44.738768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cb8d4-9a6a-4ef7-bbe1-e1a376a171ef None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:44.750608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 387/792] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Aug 9 18:30:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:44.758604 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe13cc-dff7-4f2d-8469-132f68f96378 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:44.908921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:44.923760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe13cc-dff7-4f2d-8469-132f68f96378 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:44.934595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:44.949723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cb8d4-9a6a-4ef7-bbe1-e1a376a171ef None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:44.967292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edfb0a-8de8-48a3-9611-56ec46cf19cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:44.968551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edfb0a-8de8-48a3-9611-56ec46cf19cc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:44.969016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edfb0a-8de8-48a3-9611-56ec46cf19cc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:44.969650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edfb0a-8de8-48a3-9611-56ec46cf19cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:44.983296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edfb0a-8de8-48a3-9611-56ec46cf19cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:44.985143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edfb0a-8de8-48a3-9611-56ec46cf19cc None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:45.010974 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece98244-312c-4b2e-9f41-8f7beb787cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:45.011802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece98244-312c-4b2e-9f41-8f7beb787cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:45.011916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece98244-312c-4b2e-9f41-8f7beb787cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:45.012123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece98244-312c-4b2e-9f41-8f7beb787cf8 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:45.012769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece98244-312c-4b2e-9f41-8f7beb787cf8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:45.013073 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece98244-312c-4b2e-9f41-8f7beb787cf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:45.014487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece98244-312c-4b2e-9f41-8f7beb787cf8 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-52edfb0a-8de8-48a3-9611-56ec46cf19cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:45.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:45.023052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:45.023943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:45.023943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:45.023943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:45.024124 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 406/793] 10.222.0.57 () {56 vars in 1124 bytes} [Tue Aug 9 18:30:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:45.024124 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54650b-7f63-4945-98cb-3dc837e7eefe None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:45.024707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54650b-7f63-4945-98cb-3dc837e7eefe None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:45.024707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54650b-7f63-4945-98cb-3dc837e7eefe None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:45.030093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 407/794] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:45.030093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54650b-7f63-4945-98cb-3dc837e7eefe None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:45.030772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 388/795] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:45.035296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f81d6a-d989-4f49-b753-40e14f602100 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:45.035885 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f81d6a-d989-4f49-b753-40e14f602100 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:45.036406 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f81d6a-d989-4f49-b753-40e14f602100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:45.063801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-82f81d6a-d989-4f49-b753-40e14f602100 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:45.070378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-82f81d6a-d989-4f49-b753-40e14f602100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:45.167912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 389/796] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:45.167912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f81d6a-d989-4f49-b753-40e14f602100 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:45.168157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54650b-7f63-4945-98cb-3dc837e7eefe None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:45.244405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbd3fb-ccb9-4345-aa80-1564e3881109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:45.245078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbd3fb-ccb9-4345-aa80-1564e3881109 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:45.245362 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbd3fb-ccb9-4345-aa80-1564e3881109 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:45.245663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbd3fb-ccb9-4345-aa80-1564e3881109 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:45.246334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbd3fb-ccb9-4345-aa80-1564e3881109 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:45.247726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbd3fb-ccb9-4345-aa80-1564e3881109 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:45.300454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:45.337434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f81d6a-d989-4f49-b753-40e14f602100 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:45.375772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 408/797] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:30:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:45.376771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbd3fb-ccb9-4345-aa80-1564e3881109 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:45.398677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:45.404209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb89e55-52ac-4257-9d38-c703460b2a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:45.404838 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb89e55-52ac-4257-9d38-c703460b2a63 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:45.405122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb89e55-52ac-4257-9d38-c703460b2a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:45.405378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb89e55-52ac-4257-9d38-c703460b2a63 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:45.407795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb89e55-52ac-4257-9d38-c703460b2a63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:45.409485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb89e55-52ac-4257-9d38-c703460b2a63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:45.411646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb89e55-52ac-4257-9d38-c703460b2a63 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:45.439999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 390/798] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:45.448407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb89e55-52ac-4257-9d38-c703460b2a63 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:45.638259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbd3fb-ccb9-4345-aa80-1564e3881109 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:45.695028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a6dc1-2b1c-4ab6-9dc1-a8d4c3783329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:45.695859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a6dc1-2b1c-4ab6-9dc1-a8d4c3783329 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:45.696150 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a6dc1-2b1c-4ab6-9dc1-a8d4c3783329 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:45.696424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a6dc1-2b1c-4ab6-9dc1-a8d4c3783329 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:45.709648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb89e55-52ac-4257-9d38-c703460b2a63 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:45.712389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a6dc1-2b1c-4ab6-9dc1-a8d4c3783329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-ae8a6dc1-2b1c-4ab6-9dc1-a8d4c3783329 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:45.723590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:45.724646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:45.726086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 409/799] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:45.734828 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a6dc1-2b1c-4ab6-9dc1-a8d4c3783329 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:45.739860 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:45.757345 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933e6b6-dcd5-4ef0-b1ee-3bfefd3ea608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:45.759052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933e6b6-dcd5-4ef0-b1ee-3bfefd3ea608 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:45.759485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933e6b6-dcd5-4ef0-b1ee-3bfefd3ea608 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:45.760335 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933e6b6-dcd5-4ef0-b1ee-3bfefd3ea608 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:45.780308 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933e6b6-dcd5-4ef0-b1ee-3bfefd3ea608 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:45.783107 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933e6b6-dcd5-4ef0-b1ee-3bfefd3ea608 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-7933e6b6-dcd5-4ef0-b1ee-3bfefd3ea608 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:45.812175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:45.813193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:45.814123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:45.814123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:45.814123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:45.814123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:45.814123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:45.814123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:45.814123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:45.814123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:45.818352 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 391/800] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:45.825309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933e6b6-dcd5-4ef0-b1ee-3bfefd3ea608 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:45.882481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:45.925551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a6dc1-2b1c-4ab6-9dc1-a8d4c3783329 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:45.967749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195a2c9-2978-4daa-82c5-1979ca681635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:45.968450 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195a2c9-2978-4daa-82c5-1979ca681635 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:45.973551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195a2c9-2978-4daa-82c5-1979ca681635 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:45.973551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195a2c9-2978-4daa-82c5-1979ca681635 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:45.973551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195a2c9-2978-4daa-82c5-1979ca681635 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:45.973551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195a2c9-2978-4daa-82c5-1979ca681635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:45.973551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195a2c9-2978-4daa-82c5-1979ca681635 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:45.982614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 410/801] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:45.982822 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945361b1-a8ab-4480-b8f8-84a6fafdcd07 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:45.983090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945361b1-a8ab-4480-b8f8-84a6fafdcd07 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:45.983364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945361b1-a8ab-4480-b8f8-84a6fafdcd07 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:45.984862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 411/802] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:45.986931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e302065-fd75-41d6-bdf7-89cc0c0f82e2 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:45.986931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e302065-fd75-41d6-bdf7-89cc0c0f82e2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:45.986931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e302065-fd75-41d6-bdf7-89cc0c0f82e2 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:45.988911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 412/803] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:45.991417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d0868-5656-4110-b06a-ffb73c7ee832 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:45.991551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d0868-5656-4110-b06a-ffb73c7ee832 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:45.991957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d0868-5656-4110-b06a-ffb73c7ee832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:45.997490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933e6b6-dcd5-4ef0-b1ee-3bfefd3ea608 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:46.021692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f29d0868-5656-4110-b06a-ffb73c7ee832 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:46.024367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f29d0868-5656-4110-b06a-ffb73c7ee832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:46.041828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d3bcb-969b-4c11-941a-0eaa204a52b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:46.042522 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d3bcb-969b-4c11-941a-0eaa204a52b5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:46.042831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d3bcb-969b-4c11-941a-0eaa204a52b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:46.043189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d3bcb-969b-4c11-941a-0eaa204a52b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:46.044009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d3bcb-969b-4c11-941a-0eaa204a52b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:46.044316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d3bcb-969b-4c11-941a-0eaa204a52b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:46.045793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d3bcb-969b-4c11-941a-0eaa204a52b5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:46.061574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 392/804] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:46.061914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d3bcb-969b-4c11-941a-0eaa204a52b5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:46.118908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 413/805] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:46.118908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4fa1d-6d5f-4934-90e2-0809fd0dfda5 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:46.118908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4fa1d-6d5f-4934-90e2-0809fd0dfda5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:46.118908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4fa1d-6d5f-4934-90e2-0809fd0dfda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:46.152778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:46.154373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-33c4fa1d-6d5f-4934-90e2-0809fd0dfda5 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:46.155750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-33c4fa1d-6d5f-4934-90e2-0809fd0dfda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:46.227353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:46.233314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d3bcb-969b-4c11-941a-0eaa204a52b5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:46.242196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 414/806] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:46.250516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4fa1d-6d5f-4934-90e2-0809fd0dfda5 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:46.293316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7246d-93e3-4eb8-8719-d9b56e674e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:46.294003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7246d-93e3-4eb8-8719-d9b56e674e83 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:46.294287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7246d-93e3-4eb8-8719-d9b56e674e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:46.294698 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7246d-93e3-4eb8-8719-d9b56e674e83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:46.312008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7246d-93e3-4eb8-8719-d9b56e674e83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:46.313483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7246d-93e3-4eb8-8719-d9b56e674e83 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-0ae7246d-93e3-4eb8-8719-d9b56e674e83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:46.330083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:46.331215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:46.332151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:46.332151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:46.332151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:46.336201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 393/807] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:46.337136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7246d-93e3-4eb8-8719-d9b56e674e83 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:46.407612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4fa1d-6d5f-4934-90e2-0809fd0dfda5 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:46.450009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6f396-75ed-4da1-982b-e315a9e508f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:46.451590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6f396-75ed-4da1-982b-e315a9e508f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:46.453109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6f396-75ed-4da1-982b-e315a9e508f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:46.453898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6f396-75ed-4da1-982b-e315a9e508f2 None admin] PATH_INFO: `/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/2ff735ecfcdf49089b5cfeafc56048cf/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:46.455552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:46.459503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6f396-75ed-4da1-982b-e315a9e508f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=abcccf694e7b4f66a7372e0ede416caf, user_id=2ff735ecfcdf49089b5cfeafc56048cf, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:46.460651 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7246d-93e3-4eb8-8719-d9b56e674e83 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:46.463776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6f396-75ed-4da1-982b-e315a9e508f2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:46.501855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fbfc9-ea7e-4f6e-bb00-7c8e24d1e200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:46.502421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fbfc9-ea7e-4f6e-bb00-7c8e24d1e200 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:46.502709 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fbfc9-ea7e-4f6e-bb00-7c8e24d1e200 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:46.502967 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fbfc9-ea7e-4f6e-bb00-7c8e24d1e200 None admin] PATH_INFO: `/v3/services/network` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:46.503883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fbfc9-ea7e-4f6e-bb00-7c8e24d1e200 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:46.505305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fbfc9-ea7e-4f6e-bb00-7c8e24d1e200 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-212fbfc9-ea7e-4f6e-bb00-7c8e24d1e200 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:46.514892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:46.515956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:46.516813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 394/808] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:46 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:46.517647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fbfc9-ea7e-4f6e-bb00-7c8e24d1e200 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:46.595020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 415/809] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:30:46 2022] PUT /identity/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/2ff735ecfcdf49089b5cfeafc56048cf/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:30:46.596464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6f396-75ed-4da1-982b-e315a9e508f2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:46.692197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fbfc9-ea7e-4f6e-bb00-7c8e24d1e200 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:46.699453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:46.764068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a834ced-da21-450b-873b-8bc4515aea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:46.764068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a834ced-da21-450b-873b-8bc4515aea72 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:46.764068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a834ced-da21-450b-873b-8bc4515aea72 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:46.764068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a834ced-da21-450b-873b-8bc4515aea72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:46.779035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a834ced-da21-450b-873b-8bc4515aea72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-7a834ced-da21-450b-873b-8bc4515aea72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:46.784590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:46.785605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:46.786482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 395/810] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:46.787175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6f396-75ed-4da1-982b-e315a9e508f2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:46.787248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a834ced-da21-450b-873b-8bc4515aea72 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:46.851541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81855b55-6057-4b92-9b8d-d840e55a6af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:46.852512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81855b55-6057-4b92-9b8d-d840e55a6af0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:46.852754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:46.852873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81855b55-6057-4b92-9b8d-d840e55a6af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:46.854725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81855b55-6057-4b92-9b8d-d840e55a6af0 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:46.854725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81855b55-6057-4b92-9b8d-d840e55a6af0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:46.854725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81855b55-6057-4b92-9b8d-d840e55a6af0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:46.863889 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81855b55-6057-4b92-9b8d-d840e55a6af0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:46.884289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 416/811] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:46.885443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81855b55-6057-4b92-9b8d-d840e55a6af0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:46.952643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a834ced-da21-450b-873b-8bc4515aea72 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:47.018821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333747a2-c103-4509-a1fa-25221a9c0dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:47.020731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333747a2-c103-4509-a1fa-25221a9c0dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:47.021958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333747a2-c103-4509-a1fa-25221a9c0dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:47.023584 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333747a2-c103-4509-a1fa-25221a9c0dd3 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:47.025877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333747a2-c103-4509-a1fa-25221a9c0dd3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:47.027077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333747a2-c103-4509-a1fa-25221a9c0dd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:47.031620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333747a2-c103-4509-a1fa-25221a9c0dd3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:47.046034 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 396/812] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:30:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:47.048532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333747a2-c103-4509-a1fa-25221a9c0dd3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:47.168727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81855b55-6057-4b92-9b8d-d840e55a6af0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:47.188972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:47.192549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:47.209069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333747a2-c103-4509-a1fa-25221a9c0dd3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:47.238493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7acfaa-9324-4c49-b719-78aacb450d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:47.239128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7acfaa-9324-4c49-b719-78aacb450d11 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:47.239434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7acfaa-9324-4c49-b719-78aacb450d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:47.239657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7acfaa-9324-4c49-b719-78aacb450d11 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:47.240326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7acfaa-9324-4c49-b719-78aacb450d11 None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:47.240624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7acfaa-9324-4c49-b719-78aacb450d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:47.243462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7acfaa-9324-4c49-b719-78aacb450d11 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:47.255701 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193c167-2cff-4065-af8e-f9e7cabbaae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:47.257332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193c167-2cff-4065-af8e-f9e7cabbaae3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:47.257906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193c167-2cff-4065-af8e-f9e7cabbaae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:47.258484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193c167-2cff-4065-af8e-f9e7cabbaae3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:47.262033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 417/813] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:47.268428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7acfaa-9324-4c49-b719-78aacb450d11 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:47.283772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193c167-2cff-4065-af8e-f9e7cabbaae3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:47.285891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193c167-2cff-4065-af8e-f9e7cabbaae3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-e193c167-2cff-4065-af8e-f9e7cabbaae3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:47.301136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:47.302149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:47.303059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:47.303059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:47.303059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:47.303059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:47.303059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:47.303059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:47.303059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:47.303059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:47.303059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:30:47.303059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:47.306769 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 397/814] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 9 18:30:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:47.308048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193c167-2cff-4065-af8e-f9e7cabbaae3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:47.473779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193c167-2cff-4065-af8e-f9e7cabbaae3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:47.478513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7acfaa-9324-4c49-b719-78aacb450d11 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:47.513601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf81dd7-d7b2-4886-8981-a2e56037bc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:47.514669 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf81dd7-d7b2-4886-8981-a2e56037bc81 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:47.515213 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf81dd7-d7b2-4886-8981-a2e56037bc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:47.515738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf81dd7-d7b2-4886-8981-a2e56037bc81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:47.529151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf81dd7-d7b2-4886-8981-a2e56037bc81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:47.531268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf81dd7-d7b2-4886-8981-a2e56037bc81 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:47.533548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9826e-94b0-4a65-9cd4-f4b0876c69fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:47.534146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9826e-94b0-4a65-9cd4-f4b0876c69fe None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:47.534430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9826e-94b0-4a65-9cd4-f4b0876c69fe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:47.534711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9826e-94b0-4a65-9cd4-f4b0876c69fe None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:47.535353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9826e-94b0-4a65-9cd4-f4b0876c69fe None admin] RBAC: Adding query filter params (type=network) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:47.535658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9826e-94b0-4a65-9cd4-f4b0876c69fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:47.537032 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9826e-94b0-4a65-9cd4-f4b0876c69fe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:47.541886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:47.548594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 418/815] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:30:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:47.550193 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9826e-94b0-4a65-9cd4-f4b0876c69fe None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-cdf81dd7-d7b2-4886-8981-a2e56037bc81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:47.554328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:47.562857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:47.563875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:47.563875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:47.563875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:47.563875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 398/816] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:47.563875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf81dd7-d7b2-4886-8981-a2e56037bc81 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:47.622765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:47.692360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf81dd7-d7b2-4886-8981-a2e56037bc81 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:47.723052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9826e-94b0-4a65-9cd4-f4b0876c69fe None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:47.732550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65122c00-f47c-454f-b037-60bcde125069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:47.733629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65122c00-f47c-454f-b037-60bcde125069 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:47.734011 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65122c00-f47c-454f-b037-60bcde125069 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:47.734382 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65122c00-f47c-454f-b037-60bcde125069 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:47.735519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65122c00-f47c-454f-b037-60bcde125069 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:47.737783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65122c00-f47c-454f-b037-60bcde125069 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:47.762625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbb56f-c6fc-4945-bcaa-194c8911e9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:47.763606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbb56f-c6fc-4945-bcaa-194c8911e9fe None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:47.764182 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbb56f-c6fc-4945-bcaa-194c8911e9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:47.764647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbb56f-c6fc-4945-bcaa-194c8911e9fe None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:47.765594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbb56f-c6fc-4945-bcaa-194c8911e9fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:47.766081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbb56f-c6fc-4945-bcaa-194c8911e9fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:47.767579 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbb56f-c6fc-4945-bcaa-194c8911e9fe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:47.793507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 419/817] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 9 18:30:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:47.794546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbb56f-c6fc-4945-bcaa-194c8911e9fe None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:47.842074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 399/818] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:30:47 2022] POST /identity/v3/endpoints => generated 337 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:47.845068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ae902-e71e-48a0-b40e-fa2c977165e6 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:47.845068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ae902-e71e-48a0-b40e-fa2c977165e6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:47.845068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ae902-e71e-48a0-b40e-fa2c977165e6 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:47.863836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 400/819] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:47 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:47.863836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ae902-e71e-48a0-b40e-fa2c977165e6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:47.983554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:47.994280 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbb56f-c6fc-4945-bcaa-194c8911e9fe None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:48.004511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:48.062822 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c6a9d-7b79-47dd-8684-37d0b72926fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:48.063735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c6a9d-7b79-47dd-8684-37d0b72926fe None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:48.064023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c6a9d-7b79-47dd-8684-37d0b72926fe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:48.064297 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c6a9d-7b79-47dd-8684-37d0b72926fe None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:48.064998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c6a9d-7b79-47dd-8684-37d0b72926fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:48.065304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c6a9d-7b79-47dd-8684-37d0b72926fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:48.068115 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c6a9d-7b79-47dd-8684-37d0b72926fe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:48.092952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ae902-e71e-48a0-b40e-fa2c977165e6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:48.096762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 420/820] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:48.097222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eb591-824d-41be-b918-bf7cf03f1b8a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:48.097513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eb591-824d-41be-b918-bf7cf03f1b8a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:48.101954 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eb591-824d-41be-b918-bf7cf03f1b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:48.124917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-166eb591-824d-41be-b918-bf7cf03f1b8a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:48.126701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-166eb591-824d-41be-b918-bf7cf03f1b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:48.157555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f20d2-23d9-4518-82f5-21c0b88ca5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:48.158777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f20d2-23d9-4518-82f5-21c0b88ca5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:48.159325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f20d2-23d9-4518-82f5-21c0b88ca5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:48.159609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f20d2-23d9-4518-82f5-21c0b88ca5f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:48.217257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 421/821] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:48.218455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166eb591-824d-41be-b918-bf7cf03f1b8a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:48.257150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f20d2-23d9-4518-82f5-21c0b88ca5f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:48.258751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f20d2-23d9-4518-82f5-21c0b88ca5f7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:48.339611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:48.353109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:48.382536 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166eb591-824d-41be-b918-bf7cf03f1b8a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:48.395183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 401/822] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:47 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:30:48.396199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f20d2-23d9-4518-82f5-21c0b88ca5f7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:48.424690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91780e4b-78ba-426e-933a-7b7bb94d6957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:48.425912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91780e4b-78ba-426e-933a-7b7bb94d6957 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:48.426399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91780e4b-78ba-426e-933a-7b7bb94d6957 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:48.426897 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91780e4b-78ba-426e-933a-7b7bb94d6957 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:48.440529 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91780e4b-78ba-426e-933a-7b7bb94d6957 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:48.443208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91780e4b-78ba-426e-933a-7b7bb94d6957 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-91780e4b-78ba-426e-933a-7b7bb94d6957 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:48.460630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:48.461711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:48.462644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:48.462644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:48.462644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:48.462644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:48.462644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:48.462644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:48.462644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:48.462644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:48.462644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 18:30:48.462644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:48.462644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 422/823] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 9 18:30:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:48.469250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91780e4b-78ba-426e-933a-7b7bb94d6957 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:48.597772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f20d2-23d9-4518-82f5-21c0b88ca5f7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:48.694256 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91780e4b-78ba-426e-933a-7b7bb94d6957 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:48.714827 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b05539-9acd-4088-b099-89fa2d444947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:48.716864 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b05539-9acd-4088-b099-89fa2d444947 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:48.717987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b05539-9acd-4088-b099-89fa2d444947 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:48.718891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b05539-9acd-4088-b099-89fa2d444947 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:48.721035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b05539-9acd-4088-b099-89fa2d444947 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:48.730450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b05539-9acd-4088-b099-89fa2d444947 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:48.745871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137d0ff-9bd7-41ad-91e3-abab40772fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:48.746795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137d0ff-9bd7-41ad-91e3-abab40772fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:48.747165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137d0ff-9bd7-41ad-91e3-abab40772fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:48.747589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137d0ff-9bd7-41ad-91e3-abab40772fc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:48.764469 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137d0ff-9bd7-41ad-91e3-abab40772fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:48.766732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137d0ff-9bd7-41ad-91e3-abab40772fc4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:48.769466 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-a8b05539-9acd-4088-b099-89fa2d444947 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:48.779912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:48.785561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 18:30:48.786586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-9137d0ff-9bd7-41ad-91e3-abab40772fc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:48.787513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:48.789046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:48.790377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:48.790377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:48.790377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:48.790377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 423/824] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:48.790377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137d0ff-9bd7-41ad-91e3-abab40772fc4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:48.792181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 402/825] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:30:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:48.798028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b05539-9acd-4088-b099-89fa2d444947 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:48.836443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:48.940048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b05539-9acd-4088-b099-89fa2d444947 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:48.941480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137d0ff-9bd7-41ad-91e3-abab40772fc4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:48.987683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfeb48-eb5c-41c8-a258-e7e57e6d8226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:48.988169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfeb48-eb5c-41c8-a258-e7e57e6d8226 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:48.988438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfeb48-eb5c-41c8-a258-e7e57e6d8226 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:48.988829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfeb48-eb5c-41c8-a258-e7e57e6d8226 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:48.989342 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfeb48-eb5c-41c8-a258-e7e57e6d8226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:48.989676 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfeb48-eb5c-41c8-a258-e7e57e6d8226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:48.991627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfeb48-eb5c-41c8-a258-e7e57e6d8226 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:48.997418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb4193-0ce9-491c-ba7d-4439d8ecc0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:48.998063 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb4193-0ce9-491c-ba7d-4439d8ecc0cc None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:48.998346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb4193-0ce9-491c-ba7d-4439d8ecc0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:48.998622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb4193-0ce9-491c-ba7d-4439d8ecc0cc None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:48.999357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb4193-0ce9-491c-ba7d-4439d8ecc0cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:48.999659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb4193-0ce9-491c-ba7d-4439d8ecc0cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:49.001091 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb4193-0ce9-491c-ba7d-4439d8ecc0cc None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:49.005682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 403/826] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:30:49.013508 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfeb48-eb5c-41c8-a258-e7e57e6d8226 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:49.028091 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 424/827] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Aug 9 18:30:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:49.029248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52f0a-e46b-4eb7-ba91-2adf0cdc7c5a None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:49.029559 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52f0a-e46b-4eb7-ba91-2adf0cdc7c5a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:49.029949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52f0a-e46b-4eb7-ba91-2adf0cdc7c5a None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:49.031676 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 425/828] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:49.032245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd52f0a-e46b-4eb7-ba91-2adf0cdc7c5a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:49.100848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:49.127772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:49.149602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfeb48-eb5c-41c8-a258-e7e57e6d8226 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:49.154184 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd52f0a-e46b-4eb7-ba91-2adf0cdc7c5a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:49.188799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458465ee-45fc-4fd1-866d-f2f186914edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:49.189336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3e67f-9879-44de-b7f8-3395dfd33824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:49.189458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458465ee-45fc-4fd1-866d-f2f186914edd None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:49.189702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458465ee-45fc-4fd1-866d-f2f186914edd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:49.189975 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458465ee-45fc-4fd1-866d-f2f186914edd None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:49.190424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3e67f-9879-44de-b7f8-3395dfd33824 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:49.190612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458465ee-45fc-4fd1-866d-f2f186914edd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:49.190772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3e67f-9879-44de-b7f8-3395dfd33824 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:49.190919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458465ee-45fc-4fd1-866d-f2f186914edd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:49.191172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3e67f-9879-44de-b7f8-3395dfd33824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:49.192279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458465ee-45fc-4fd1-866d-f2f186914edd None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:49.197453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 404/829] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:30:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:49.198251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458465ee-45fc-4fd1-866d-f2f186914edd None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:49.199220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3e67f-9879-44de-b7f8-3395dfd33824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:49.200760 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3e67f-9879-44de-b7f8-3395dfd33824 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-8cc3e67f-9879-44de-b7f8-3395dfd33824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:49.212365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:49.213437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:49.214675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:49.214675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:49.214675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:49.214675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:49.214675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:49.214675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:49.214675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:49.214675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:49.214675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 426/830] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:49.214675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad7c25-7171-4518-98af-227e5c151af6 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:49.215331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad7c25-7171-4518-98af-227e5c151af6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:49.215331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad7c25-7171-4518-98af-227e5c151af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:49.238948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-05ad7c25-7171-4518-98af-227e5c151af6 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:49.240166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-05ad7c25-7171-4518-98af-227e5c151af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:49.327322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 427/831] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:49.328754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad7c25-7171-4518-98af-227e5c151af6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:49.352798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458465ee-45fc-4fd1-866d-f2f186914edd None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:49.392313 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:49.406847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b86acd-faed-4284-b409-9af7fbc580c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:49.407672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b86acd-faed-4284-b409-9af7fbc580c0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:49.408126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b86acd-faed-4284-b409-9af7fbc580c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:49.408580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b86acd-faed-4284-b409-9af7fbc580c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:49.425464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b86acd-faed-4284-b409-9af7fbc580c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:49.428026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b86acd-faed-4284-b409-9af7fbc580c0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:49.431383 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-b7b86acd-faed-4284-b409-9af7fbc580c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:49.451871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:49.452951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:49.453980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:49.453980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:49.453980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:49.453980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 405/832] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:49.454716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b86acd-faed-4284-b409-9af7fbc580c0 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:49.478403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad7c25-7171-4518-98af-227e5c151af6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:49.514419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3554d-18f5-4814-87de-45cb7b9bc4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:49.515044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3554d-18f5-4814-87de-45cb7b9bc4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:49.515325 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3554d-18f5-4814-87de-45cb7b9bc4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:49.515629 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3554d-18f5-4814-87de-45cb7b9bc4b2 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:49.516272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3554d-18f5-4814-87de-45cb7b9bc4b2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:49.516579 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3554d-18f5-4814-87de-45cb7b9bc4b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:49.517984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3554d-18f5-4814-87de-45cb7b9bc4b2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:49.522869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 428/833] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:30:49 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:49.523462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3554d-18f5-4814-87de-45cb7b9bc4b2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:49.572649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b86acd-faed-4284-b409-9af7fbc580c0 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:49.618055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27fc8c-168f-40ec-8242-236cfc41daf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:49.618736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27fc8c-168f-40ec-8242-236cfc41daf2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:49.619013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27fc8c-168f-40ec-8242-236cfc41daf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:49.619292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27fc8c-168f-40ec-8242-236cfc41daf2 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:49.620001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27fc8c-168f-40ec-8242-236cfc41daf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:49.620312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27fc8c-168f-40ec-8242-236cfc41daf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:49.621879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27fc8c-168f-40ec-8242-236cfc41daf2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:49.642419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 406/834] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:49.643103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27fc8c-168f-40ec-8242-236cfc41daf2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:49.660456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3554d-18f5-4814-87de-45cb7b9bc4b2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:49.670010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:49.705841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde6ae3-e6bd-4293-9334-5df16dd27123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:49.706471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde6ae3-e6bd-4293-9334-5df16dd27123 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:49.706660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde6ae3-e6bd-4293-9334-5df16dd27123 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:49.706931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde6ae3-e6bd-4293-9334-5df16dd27123 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:49.708582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:49.719182 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde6ae3-e6bd-4293-9334-5df16dd27123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-fbde6ae3-e6bd-4293-9334-5df16dd27123 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:49.724404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:49.726639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 429/835] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:49.728085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde6ae3-e6bd-4293-9334-5df16dd27123 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:49.762303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27fc8c-168f-40ec-8242-236cfc41daf2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:49.799653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053a784-452d-49b3-a3fd-80d7c8ff443b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:49.800668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053a784-452d-49b3-a3fd-80d7c8ff443b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:49.801177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053a784-452d-49b3-a3fd-80d7c8ff443b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:49.801659 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053a784-452d-49b3-a3fd-80d7c8ff443b None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:49.802548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053a784-452d-49b3-a3fd-80d7c8ff443b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:49.803061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053a784-452d-49b3-a3fd-80d7c8ff443b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:49.805473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053a784-452d-49b3-a3fd-80d7c8ff443b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:49.819242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 407/836] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:49.821203 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053a784-452d-49b3-a3fd-80d7c8ff443b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:49.845801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde6ae3-e6bd-4293-9334-5df16dd27123 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:49.882434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad3cbc-514a-4265-93cb-d8c9a4419507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:49.883045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad3cbc-514a-4265-93cb-d8c9a4419507 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:49.883326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad3cbc-514a-4265-93cb-d8c9a4419507 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:49.883596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad3cbc-514a-4265-93cb-d8c9a4419507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:49.884350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad3cbc-514a-4265-93cb-d8c9a4419507 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=e2ecd7d1cce644b0982cc23740537933) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:49.884665 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad3cbc-514a-4265-93cb-d8c9a4419507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:49.886126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad3cbc-514a-4265-93cb-d8c9a4419507 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:49.901552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 430/837] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:49 2022] GET /identity/v3/role_assignments?user.id=e2ecd7d1cce644b0982cc23740537933&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:49.901552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad3cbc-514a-4265-93cb-d8c9a4419507 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:49.935732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:49.953805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053a784-452d-49b3-a3fd-80d7c8ff443b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:49.972294 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:50.000536 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77f33d-3706-4332-888f-178666eb59d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:50.001171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77f33d-3706-4332-888f-178666eb59d1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:50.001451 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77f33d-3706-4332-888f-178666eb59d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:50.001904 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77f33d-3706-4332-888f-178666eb59d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:50.002546 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77f33d-3706-4332-888f-178666eb59d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:50.002852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77f33d-3706-4332-888f-178666eb59d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:50.004378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77f33d-3706-4332-888f-178666eb59d1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:50.016130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 408/838] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:30:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:50.021504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77f33d-3706-4332-888f-178666eb59d1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:50.034747 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad3cbc-514a-4265-93cb-d8c9a4419507 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:50.076644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51312327-8458-49ea-b2cf-11272adce4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:50.077282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51312327-8458-49ea-b2cf-11272adce4ae None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:50.077588 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51312327-8458-49ea-b2cf-11272adce4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:50.077876 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51312327-8458-49ea-b2cf-11272adce4ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:50.087056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51312327-8458-49ea-b2cf-11272adce4ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:50.088507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51312327-8458-49ea-b2cf-11272adce4ae None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-51312327-8458-49ea-b2cf-11272adce4ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:50.103098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:50.106323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:50.107208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:50.107208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:50.107208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:50.107208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:50.107208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:50.107208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:50.107208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:50.107208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:50.107208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 431/839] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:50.110292 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51312327-8458-49ea-b2cf-11272adce4ae None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:50.154909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77f33d-3706-4332-888f-178666eb59d1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:50.213964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c32c8-e4bb-4f4e-8809-9c0e7fe47f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:50.214844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c32c8-e4bb-4f4e-8809-9c0e7fe47f76 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:50.215126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c32c8-e4bb-4f4e-8809-9c0e7fe47f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:50.215768 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c32c8-e4bb-4f4e-8809-9c0e7fe47f76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:50.221516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:50.229300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c32c8-e4bb-4f4e-8809-9c0e7fe47f76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:50.230727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c32c8-e4bb-4f4e-8809-9c0e7fe47f76 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:50.231498 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51312327-8458-49ea-b2cf-11272adce4ae None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-038c32c8-e4bb-4f4e-8809-9c0e7fe47f76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:50.245763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:50.246882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:50.247740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:50.247740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:50.247740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:50.247740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 409/840] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:50.252531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c32c8-e4bb-4f4e-8809-9c0e7fe47f76 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:50.284078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82f7c9-2fd0-4491-9230-3764691974ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:50.285005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82f7c9-2fd0-4491-9230-3764691974ae None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:50.285449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82f7c9-2fd0-4491-9230-3764691974ae None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:50.285924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82f7c9-2fd0-4491-9230-3764691974ae None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:50.286853 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82f7c9-2fd0-4491-9230-3764691974ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:50.287350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82f7c9-2fd0-4491-9230-3764691974ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:50.288951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82f7c9-2fd0-4491-9230-3764691974ae None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:50.293507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:50.307920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 432/841] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:50.314210 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82f7c9-2fd0-4491-9230-3764691974ae None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:50.384287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c32c8-e4bb-4f4e-8809-9c0e7fe47f76 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:50.422894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371dda3-7232-4738-914f-2d001ebfcb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:50.424114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371dda3-7232-4738-914f-2d001ebfcb36 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:50.424807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371dda3-7232-4738-914f-2d001ebfcb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:50.425222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371dda3-7232-4738-914f-2d001ebfcb36 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:50.426291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371dda3-7232-4738-914f-2d001ebfcb36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:50.427134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371dda3-7232-4738-914f-2d001ebfcb36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:50.429458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371dda3-7232-4738-914f-2d001ebfcb36 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:50.433180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82f7c9-2fd0-4491-9230-3764691974ae None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:50.445295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 410/842] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:50.450234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371dda3-7232-4738-914f-2d001ebfcb36 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:50.477162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104035-9f14-43de-8b10-8261e68eb71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:50.478616 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104035-9f14-43de-8b10-8261e68eb71a None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:50.478616 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104035-9f14-43de-8b10-8261e68eb71a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:50.478616 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104035-9f14-43de-8b10-8261e68eb71a None admin] PATH_INFO: `/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/1255cb6e360244ba9d5cec5cc65a1f8a/roles/106b56dfde9d4c4f9f32ca4bec0b3e87` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:50.502228 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78104035-9f14-43de-8b10-8261e68eb71a None admin] RBAC: Authorizing `identity:create_grant(project_id=abcccf694e7b4f66a7372e0ede416caf, user_id=1255cb6e360244ba9d5cec5cc65a1f8a, role_id=106b56dfde9d4c4f9f32ca4bec0b3e87)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:50.503748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78104035-9f14-43de-8b10-8261e68eb71a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:50.564189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:50.569322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:50.580144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 433/843] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:30:50 2022] PUT /identity/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/1255cb6e360244ba9d5cec5cc65a1f8a/roles/106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:30:50.599830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371dda3-7232-4738-914f-2d001ebfcb36 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:50.646429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a8ddf-05a0-4ff4-abab-452f76ca3d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:50.647988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a8ddf-05a0-4ff4-abab-452f76ca3d9d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:50.648556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a8ddf-05a0-4ff4-abab-452f76ca3d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:50.649043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a8ddf-05a0-4ff4-abab-452f76ca3d9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:50.659700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a8ddf-05a0-4ff4-abab-452f76ca3d9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:50.661584 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a8ddf-05a0-4ff4-abab-452f76ca3d9d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-a55a8ddf-05a0-4ff4-abab-452f76ca3d9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:50.675560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:50.676936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:50.678101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:50.678101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:50.678101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:50.678101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:50.678101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:50.678101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:50.678101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:50.678101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:50.678101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 18:30:50.678101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:50.681550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 411/844] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:30:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:50.687188 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104035-9f14-43de-8b10-8261e68eb71a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:50.811018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104035-9f14-43de-8b10-8261e68eb71a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:50.863979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a89f-b20b-4bb7-8725-28c9bb4c5c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:50.864590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a89f-b20b-4bb7-8725-28c9bb4c5c6c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:50.864857 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a89f-b20b-4bb7-8725-28c9bb4c5c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:50.865130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a89f-b20b-4bb7-8725-28c9bb4c5c6c None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:50.865949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a89f-b20b-4bb7-8725-28c9bb4c5c6c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:50.866254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a89f-b20b-4bb7-8725-28c9bb4c5c6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:50.867679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a89f-b20b-4bb7-8725-28c9bb4c5c6c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:50.893263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 434/845] 10.222.0.57 () {56 vars in 1135 bytes} [Tue Aug 9 18:30:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:50.902594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a8ddf-05a0-4ff4-abab-452f76ca3d9d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:51.045590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a8ddf-05a0-4ff4-abab-452f76ca3d9d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:51.085890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b8a2d-b864-4261-858d-07e683fc5e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:51.086427 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b8a2d-b864-4261-858d-07e683fc5e5a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:51.086713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b8a2d-b864-4261-858d-07e683fc5e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:51.086990 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b8a2d-b864-4261-858d-07e683fc5e5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:51.109949 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:51.157973 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b8a2d-b864-4261-858d-07e683fc5e5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:51.159440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b8a2d-b864-4261-858d-07e683fc5e5a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:51.269881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 412/846] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:30:51.276235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a89f-b20b-4bb7-8725-28c9bb4c5c6c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:51.346128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:51.381436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a89f-b20b-4bb7-8725-28c9bb4c5c6c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:51.416387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6612db-d1dc-4e4a-9a88-0ebaad39b662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:51.417017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6612db-d1dc-4e4a-9a88-0ebaad39b662 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:51.417333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6612db-d1dc-4e4a-9a88-0ebaad39b662 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:51.417604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6612db-d1dc-4e4a-9a88-0ebaad39b662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:51.429301 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6612db-d1dc-4e4a-9a88-0ebaad39b662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:51.430797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6612db-d1dc-4e4a-9a88-0ebaad39b662 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-4c6612db-d1dc-4e4a-9a88-0ebaad39b662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:51.445941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:51.446937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:51.447821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:51.447821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:51.447821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:51.447821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 435/847] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:51.453099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b8a2d-b864-4261-858d-07e683fc5e5a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:51.504136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:51.556498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b8a2d-b864-4261-858d-07e683fc5e5a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:51.591128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504cf75-1198-411c-a2f1-254409e70539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:51.591731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504cf75-1198-411c-a2f1-254409e70539 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:51.592013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504cf75-1198-411c-a2f1-254409e70539 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:51.592291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504cf75-1198-411c-a2f1-254409e70539 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:51.592941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504cf75-1198-411c-a2f1-254409e70539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:51.593245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504cf75-1198-411c-a2f1-254409e70539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:51.594644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504cf75-1198-411c-a2f1-254409e70539 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:51.607256 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 413/848] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:51.612738 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6612db-d1dc-4e4a-9a88-0ebaad39b662 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:51.680277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275d992-af24-41e5-add5-c1e4eea09905 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:51.680601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275d992-af24-41e5-add5-c1e4eea09905 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:51.680814 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275d992-af24-41e5-add5-c1e4eea09905 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:51.683058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 414/849] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:30:51.687215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbc220-dcc6-4c10-b3ea-f7641c6e39db None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:51.687656 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbc220-dcc6-4c10-b3ea-f7641c6e39db None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:51.688055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbc220-dcc6-4c10-b3ea-f7641c6e39db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:51.707430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-26dbc220-dcc6-4c10-b3ea-f7641c6e39db None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:51.708614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-26dbc220-dcc6-4c10-b3ea-f7641c6e39db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:51.726089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6612db-d1dc-4e4a-9a88-0ebaad39b662 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:51.742152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:51.770716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c4485-be2f-47d2-ad3e-dc76c791fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:51.771658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c4485-be2f-47d2-ad3e-dc76c791fc2e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:51.772880 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c4485-be2f-47d2-ad3e-dc76c791fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:51.773517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c4485-be2f-47d2-ad3e-dc76c791fc2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:51.781551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c4485-be2f-47d2-ad3e-dc76c791fc2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:51.783150 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c4485-be2f-47d2-ad3e-dc76c791fc2e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:51.785875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 415/850] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-608c4485-be2f-47d2-ad3e-dc76c791fc2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:51.795334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:51.796397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:51.797552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:51.797552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:51.797552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:51.797552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:51.797552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:51.797552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:51.797552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:51.797552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:51.799156 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 436/851] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:51.805553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbc220-dcc6-4c10-b3ea-f7641c6e39db None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:51.805781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c4485-be2f-47d2-ad3e-dc76c791fc2e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:51.856566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:51.930201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c4485-be2f-47d2-ad3e-dc76c791fc2e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:51.932527 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbc220-dcc6-4c10-b3ea-f7641c6e39db None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:51.966718 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105d082-6cd3-456b-aa27-426d9a66e2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:51.967604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105d082-6cd3-456b-aa27-426d9a66e2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:51.968031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105d082-6cd3-456b-aa27-426d9a66e2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:51.968443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105d082-6cd3-456b-aa27-426d9a66e2a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:51.969267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105d082-6cd3-456b-aa27-426d9a66e2a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:51.969746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105d082-6cd3-456b-aa27-426d9a66e2a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:51.971329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105d082-6cd3-456b-aa27-426d9a66e2a6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:51.978100 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e95374-6316-46f5-8330-f272c6601bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:51.978829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e95374-6316-46f5-8330-f272c6601bc2 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:51.979214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e95374-6316-46f5-8330-f272c6601bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:51.979591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e95374-6316-46f5-8330-f272c6601bc2 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:51.980956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e95374-6316-46f5-8330-f272c6601bc2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:51.982478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e95374-6316-46f5-8330-f272c6601bc2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:51.987305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 437/852] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:30:51.993947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105d082-6cd3-456b-aa27-426d9a66e2a6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:52.033588 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:52.050544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 416/853] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:30:51 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:52.127217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105d082-6cd3-456b-aa27-426d9a66e2a6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:52.138220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858fd45-61b0-4deb-a817-c49210479d20 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:52.138873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858fd45-61b0-4deb-a817-c49210479d20 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:52.139410 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858fd45-61b0-4deb-a817-c49210479d20 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:52.141513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 417/854] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:52.152092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46727bd6-1b4a-44d5-b2cc-31f19372fed4 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:52.152727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46727bd6-1b4a-44d5-b2cc-31f19372fed4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:52.153467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46727bd6-1b4a-44d5-b2cc-31f19372fed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:52.174138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b9a35-eed4-4d4c-ad93-ad34fee78b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:52.174757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b9a35-eed4-4d4c-ad93-ad34fee78b73 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:52.175097 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b9a35-eed4-4d4c-ad93-ad34fee78b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:52.175376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b9a35-eed4-4d4c-ad93-ad34fee78b73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:52.176153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b9a35-eed4-4d4c-ad93-ad34fee78b73 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=2ff735ecfcdf49089b5cfeafc56048cf) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:52.176419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b9a35-eed4-4d4c-ad93-ad34fee78b73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:52.177911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b9a35-eed4-4d4c-ad93-ad34fee78b73 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:52.186068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-46727bd6-1b4a-44d5-b2cc-31f19372fed4 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:52.188191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-46727bd6-1b4a-44d5-b2cc-31f19372fed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:52.194491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 438/855] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:51 2022] GET /identity/v3/role_assignments?user.id=2ff735ecfcdf49089b5cfeafc56048cf&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:30:52.282808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 418/856] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:30:52.307765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b9a35-eed4-4d4c-ad93-ad34fee78b73 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:52.408159 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:52.419160 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b9a35-eed4-4d4c-ad93-ad34fee78b73 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:52.457544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39dc58-c460-43e2-b25e-d9771c85742e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:52.458557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39dc58-c460-43e2-b25e-d9771c85742e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:52.459064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39dc58-c460-43e2-b25e-d9771c85742e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:52.459692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39dc58-c460-43e2-b25e-d9771c85742e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:52.460515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39dc58-c460-43e2-b25e-d9771c85742e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:52.462183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39dc58-c460-43e2-b25e-d9771c85742e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-9a39dc58-c460-43e2-b25e-d9771c85742e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:52.471662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:52.472635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 18:30:52.473554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:52.476819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 439/857] 10.222.0.57 () {56 vars in 1132 bytes} [Tue Aug 9 18:30:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:52.486636 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46727bd6-1b4a-44d5-b2cc-31f19372fed4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:52.515373 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:52.594086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46727bd6-1b4a-44d5-b2cc-31f19372fed4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:52.632936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22500e88-fa38-4c3b-891a-7ff6eb1526eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:52.633946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22500e88-fa38-4c3b-891a-7ff6eb1526eb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:52.634455 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22500e88-fa38-4c3b-891a-7ff6eb1526eb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:52.634940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22500e88-fa38-4c3b-891a-7ff6eb1526eb None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:52.635801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22500e88-fa38-4c3b-891a-7ff6eb1526eb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:52.636316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22500e88-fa38-4c3b-891a-7ff6eb1526eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:52.637914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22500e88-fa38-4c3b-891a-7ff6eb1526eb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:52.643404 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 419/858] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 9 18:30:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:52.649282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39dc58-c460-43e2-b25e-d9771c85742e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:52.756489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39dc58-c460-43e2-b25e-d9771c85742e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:52.791111 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938902d-12a4-4cc9-be42-cb308812b518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:52.791716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938902d-12a4-4cc9-be42-cb308812b518 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:52.791995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938902d-12a4-4cc9-be42-cb308812b518 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:52.792265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938902d-12a4-4cc9-be42-cb308812b518 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:52.792899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938902d-12a4-4cc9-be42-cb308812b518 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:52.793198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938902d-12a4-4cc9-be42-cb308812b518 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:52.794586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938902d-12a4-4cc9-be42-cb308812b518 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:52.799721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 440/859] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 9 18:30:52 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:52.835945 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54f6fb-2d41-4c1e-b275-f89530a78ac5 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:52.836238 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54f6fb-2d41-4c1e-b275-f89530a78ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:52.836498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54f6fb-2d41-4c1e-b275-f89530a78ac5 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:52.839340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 420/860] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:52.843962 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68367782-19be-444a-9561-8baef1668fae None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:52.844248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68367782-19be-444a-9561-8baef1668fae None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:52.844526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68367782-19be-444a-9561-8baef1668fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:52.864364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-68367782-19be-444a-9561-8baef1668fae None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:52.865610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-68367782-19be-444a-9561-8baef1668fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:52.880706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:52.945641 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 441/861] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:52.965196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54f6fb-2d41-4c1e-b275-f89530a78ac5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:53.068197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:53.079929 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54f6fb-2d41-4c1e-b275-f89530a78ac5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:53.117373 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c15e02-2fbe-4e6b-ad44-388e9304e46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:53.118083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c15e02-2fbe-4e6b-ad44-388e9304e46f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:53.118363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c15e02-2fbe-4e6b-ad44-388e9304e46f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:53.118627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c15e02-2fbe-4e6b-ad44-388e9304e46f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:53.129109 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c15e02-2fbe-4e6b-ad44-388e9304e46f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-51c15e02-2fbe-4e6b-ad44-388e9304e46f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:53.135223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:53.136215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:53.137066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 421/862] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:30:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:53.142983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68367782-19be-444a-9561-8baef1668fae None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:53.248888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68367782-19be-444a-9561-8baef1668fae None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:53.285271 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bf299-7c5a-4d88-9fa0-d78c2b19df95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:53.285913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bf299-7c5a-4d88-9fa0-d78c2b19df95 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:53.286195 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bf299-7c5a-4d88-9fa0-d78c2b19df95 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:53.286467 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bf299-7c5a-4d88-9fa0-d78c2b19df95 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:53.287133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bf299-7c5a-4d88-9fa0-d78c2b19df95 None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:53.287442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bf299-7c5a-4d88-9fa0-d78c2b19df95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:53.289356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bf299-7c5a-4d88-9fa0-d78c2b19df95 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:53.306622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 442/863] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:30:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:53.314030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c15e02-2fbe-4e6b-ad44-388e9304e46f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:53.435113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c15e02-2fbe-4e6b-ad44-388e9304e46f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:53.470983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facc4d4-a3b4-4c1c-b6b6-9b333b49c6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:53.471667 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facc4d4-a3b4-4c1c-b6b6-9b333b49c6da None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:53.471947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facc4d4-a3b4-4c1c-b6b6-9b333b49c6da None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:53.472267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facc4d4-a3b4-4c1c-b6b6-9b333b49c6da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:53.487531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2facc4d4-a3b4-4c1c-b6b6-9b333b49c6da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:53.489012 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2facc4d4-a3b4-4c1c-b6b6-9b333b49c6da None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-2facc4d4-a3b4-4c1c-b6b6-9b333b49c6da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:53.511284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:53.512372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:53.513258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:53.513258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:53.513258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:53.513258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 422/864] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:53.518517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bf299-7c5a-4d88-9fa0-d78c2b19df95 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:53.545879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:53.630129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bf299-7c5a-4d88-9fa0-d78c2b19df95 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:53.666845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189551e-2ccf-4098-9449-5b110796289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:53.667468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189551e-2ccf-4098-9449-5b110796289f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:53.667748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189551e-2ccf-4098-9449-5b110796289f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:53.668018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189551e-2ccf-4098-9449-5b110796289f None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:53.668678 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189551e-2ccf-4098-9449-5b110796289f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:53.669019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189551e-2ccf-4098-9449-5b110796289f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:53.670507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189551e-2ccf-4098-9449-5b110796289f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:53.690220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 443/865] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:53.698882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facc4d4-a3b4-4c1c-b6b6-9b333b49c6da None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:53.703707 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:53.808295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facc4d4-a3b4-4c1c-b6b6-9b333b49c6da None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:53.843489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0fd07-183e-45d5-bbad-c32a1e31a1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:53.844126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0fd07-183e-45d5-bbad-c32a1e31a1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:53.844407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0fd07-183e-45d5-bbad-c32a1e31a1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:53.844679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0fd07-183e-45d5-bbad-c32a1e31a1e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:53.853171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0fd07-183e-45d5-bbad-c32a1e31a1e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:53.854644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0fd07-183e-45d5-bbad-c32a1e31a1e1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-26c0fd07-183e-45d5-bbad-c32a1e31a1e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:53.866023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:53.866985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:53.867915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:53.867915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:53.867915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:53.867915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:53.867915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:53.867915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:53.867915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:53.867915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:53.867915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 18:30:53.867915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:53.867915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 423/866] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 9 18:30:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:53.874881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189551e-2ccf-4098-9449-5b110796289f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:53.958662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:53.990639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189551e-2ccf-4098-9449-5b110796289f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:54.026967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7ecda-95c2-43ac-a648-7122de931015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:54.027588 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7ecda-95c2-43ac-a648-7122de931015 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:54.027867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7ecda-95c2-43ac-a648-7122de931015 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:54.028138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7ecda-95c2-43ac-a648-7122de931015 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:54.028875 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7ecda-95c2-43ac-a648-7122de931015 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:54.029180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7ecda-95c2-43ac-a648-7122de931015 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:54.030666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7ecda-95c2-43ac-a648-7122de931015 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:54.054122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 444/867] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Aug 9 18:30:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:54.059854 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0fd07-183e-45d5-bbad-c32a1e31a1e1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:54.140004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:54.179852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0fd07-183e-45d5-bbad-c32a1e31a1e1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:54.220896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c3034-c3ae-4da7-81f2-1d6664f6179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:54.221493 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c3034-c3ae-4da7-81f2-1d6664f6179a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:54.221805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c3034-c3ae-4da7-81f2-1d6664f6179a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:54.222078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c3034-c3ae-4da7-81f2-1d6664f6179a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:54.278422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a882a-db40-4e53-9664-7d39683af99c None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:54.278986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a882a-db40-4e53-9664-7d39683af99c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:54.279476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a882a-db40-4e53-9664-7d39683af99c None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:54.280995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 445/868] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:54.290661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ab6d5-eca2-4ee2-a6ff-5c0f5c71e414 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:54.291173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ab6d5-eca2-4ee2-a6ff-5c0f5c71e414 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:54.291647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ab6d5-eca2-4ee2-a6ff-5c0f5c71e414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:54.296134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c3034-c3ae-4da7-81f2-1d6664f6179a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:54.297597 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c3034-c3ae-4da7-81f2-1d6664f6179a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:54.316459 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-dc6ab6d5-eca2-4ee2-a6ff-5c0f5c71e414 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:54.317993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dc6ab6d5-eca2-4ee2-a6ff-5c0f5c71e414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:54.366834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:54.406378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 446/869] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:54 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:54.407040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 424/870] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:54 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:30:54.413000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ab6d5-eca2-4ee2-a6ff-5c0f5c71e414 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:54.429517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c3034-c3ae-4da7-81f2-1d6664f6179a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:54.492206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:54.535852 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ab6d5-eca2-4ee2-a6ff-5c0f5c71e414 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:54.536082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c3034-c3ae-4da7-81f2-1d6664f6179a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:54.572584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540fde6-d253-4ca6-abb0-c99e717af206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:54.572990 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117ae27-01e5-47d3-8de4-39a792afa70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:54.573217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540fde6-d253-4ca6-abb0-c99e717af206 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:54.573498 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540fde6-d253-4ca6-abb0-c99e717af206 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:54.573680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117ae27-01e5-47d3-8de4-39a792afa70d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:54.573790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540fde6-d253-4ca6-abb0-c99e717af206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:54.573955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117ae27-01e5-47d3-8de4-39a792afa70d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:54.574194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117ae27-01e5-47d3-8de4-39a792afa70d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:54.574774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117ae27-01e5-47d3-8de4-39a792afa70d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:54.576137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117ae27-01e5-47d3-8de4-39a792afa70d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:54.584878 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540fde6-d253-4ca6-abb0-c99e717af206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-7117ae27-01e5-47d3-8de4-39a792afa70d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:54.585024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:54.585970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:54.591936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 425/871] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:30:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:30:54.592777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540fde6-d253-4ca6-abb0-c99e717af206 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:54.592851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618a594-53eb-4efc-95b4-cc9d2bf5f8f2 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:54.592851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618a594-53eb-4efc-95b4-cc9d2bf5f8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:54.592851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618a594-53eb-4efc-95b4-cc9d2bf5f8f2 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:54.592851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 426/872] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:54.592851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618a594-53eb-4efc-95b4-cc9d2bf5f8f2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-4540fde6-d253-4ca6-abb0-c99e717af206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:54.609457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:54.610493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:54.617102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:54.617102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:54.617102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:54.617102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 447/873] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:54.617102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b30d7-3694-49e5-b3cb-2ec0756b8dfa None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:54.617102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b30d7-3694-49e5-b3cb-2ec0756b8dfa None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:54.617102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b30d7-3694-49e5-b3cb-2ec0756b8dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:54.624627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:54.631044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2b5b30d7-3694-49e5-b3cb-2ec0756b8dfa None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:54.634641 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2b5b30d7-3694-49e5-b3cb-2ec0756b8dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:54.704926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618a594-53eb-4efc-95b4-cc9d2bf5f8f2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:54.709743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 448/874] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:54 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:54.709743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b30d7-3694-49e5-b3cb-2ec0756b8dfa None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:54.752767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a901c8-825a-42bf-94ba-044a402a3d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:54.753110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:54.753384 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a901c8-825a-42bf-94ba-044a402a3d68 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:54.753599 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a901c8-825a-42bf-94ba-044a402a3d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:54.753875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a901c8-825a-42bf-94ba-044a402a3d68 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:54.754520 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a901c8-825a-42bf-94ba-044a402a3d68 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:54.754822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a901c8-825a-42bf-94ba-044a402a3d68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:54.756214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a901c8-825a-42bf-94ba-044a402a3d68 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:54.765477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 427/875] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:30:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:54.765477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a901c8-825a-42bf-94ba-044a402a3d68 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:54.821415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b30d7-3694-49e5-b3cb-2ec0756b8dfa None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:54.858890 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068631ef-2c1c-4bc8-bcf6-cda2f95120c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:54.859927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068631ef-2c1c-4bc8-bcf6-cda2f95120c4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:54.860227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068631ef-2c1c-4bc8-bcf6-cda2f95120c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:54.860503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068631ef-2c1c-4bc8-bcf6-cda2f95120c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:54.861153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068631ef-2c1c-4bc8-bcf6-cda2f95120c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:54.861461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068631ef-2c1c-4bc8-bcf6-cda2f95120c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:54.862888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068631ef-2c1c-4bc8-bcf6-cda2f95120c4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:54.863597 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:54.873106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a901c8-825a-42bf-94ba-044a402a3d68 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:54.876446 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 449/876] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:54.877453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068631ef-2c1c-4bc8-bcf6-cda2f95120c4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:54.913909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575c545-e2e9-42cd-9ebf-1406be4bf9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:54.914569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575c545-e2e9-42cd-9ebf-1406be4bf9ae None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:54.914819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575c545-e2e9-42cd-9ebf-1406be4bf9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:54.915092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575c545-e2e9-42cd-9ebf-1406be4bf9ae None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:54.915686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575c545-e2e9-42cd-9ebf-1406be4bf9ae None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:54.917054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575c545-e2e9-42cd-9ebf-1406be4bf9ae None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-3575c545-e2e9-42cd-9ebf-1406be4bf9ae None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:54.926166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:54.927234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:54.934302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 428/877] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 9 18:30:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:54.935108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575c545-e2e9-42cd-9ebf-1406be4bf9ae None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:55.000952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:55.004792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068631ef-2c1c-4bc8-bcf6-cda2f95120c4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:55.043020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1311eb-f65c-4dcc-b7a8-69ce0ba411db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:55.043621 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1311eb-f65c-4dcc-b7a8-69ce0ba411db None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:55.043897 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1311eb-f65c-4dcc-b7a8-69ce0ba411db None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:55.044168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1311eb-f65c-4dcc-b7a8-69ce0ba411db None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:55.044345 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575c545-e2e9-42cd-9ebf-1406be4bf9ae None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:55.044797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1311eb-f65c-4dcc-b7a8-69ce0ba411db None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:55.045099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1311eb-f65c-4dcc-b7a8-69ce0ba411db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:55.046560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1311eb-f65c-4dcc-b7a8-69ce0ba411db None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:55.054687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 450/878] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:30:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:55.055701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1311eb-f65c-4dcc-b7a8-69ce0ba411db None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:55.085712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80d6c2-37ae-444b-a4f1-13d3c7a1718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:55.086364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80d6c2-37ae-444b-a4f1-13d3c7a1718b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:55.086645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80d6c2-37ae-444b-a4f1-13d3c7a1718b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:55.086919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80d6c2-37ae-444b-a4f1-13d3c7a1718b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:55.094399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80d6c2-37ae-444b-a4f1-13d3c7a1718b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:55.095368 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80d6c2-37ae-444b-a4f1-13d3c7a1718b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-4c80d6c2-37ae-444b-a4f1-13d3c7a1718b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:30:55.107765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:55.108712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:55.109868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:30:55.109868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:30:55.109868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:30:55.109868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:30:55.109868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:30:55.109868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:30:55.109868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:30:55.109868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:55.109868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 429/879] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:30:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:55.110581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c485865-de47-4bec-abaf-958c74d6dfb9 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:55.110865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c485865-de47-4bec-abaf-958c74d6dfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:55.111139 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c485865-de47-4bec-abaf-958c74d6dfb9 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:55.112411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 430/880] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:55.113424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c485865-de47-4bec-abaf-958c74d6dfb9 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:55.132273 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:55.178417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1311eb-f65c-4dcc-b7a8-69ce0ba411db None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:55.215560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a67fd-ed9d-46d9-923c-36a5bb002ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:55.216162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a67fd-ed9d-46d9-923c-36a5bb002ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:55.216445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a67fd-ed9d-46d9-923c-36a5bb002ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:55.216724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a67fd-ed9d-46d9-923c-36a5bb002ca0 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:55.217279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c485865-de47-4bec-abaf-958c74d6dfb9 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:55.217757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a67fd-ed9d-46d9-923c-36a5bb002ca0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:55.217757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a67fd-ed9d-46d9-923c-36a5bb002ca0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:55.219050 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a67fd-ed9d-46d9-923c-36a5bb002ca0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:55.223917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 451/881] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 9 18:30:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:55.225756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a67fd-ed9d-46d9-923c-36a5bb002ca0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:55.251168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51730d-21bf-4757-9a86-c2b8be482b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:55.251763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51730d-21bf-4757-9a86-c2b8be482b47 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:55.252044 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51730d-21bf-4757-9a86-c2b8be482b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:55.252310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51730d-21bf-4757-9a86-c2b8be482b47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:55.252997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51730d-21bf-4757-9a86-c2b8be482b47 None admin] RBAC: Adding query filter params (interface=public, service_id=fc000368006949e0ae12d01e5be747fc, region_id=RegionOne) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:55.253299 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51730d-21bf-4757-9a86-c2b8be482b47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:55.254710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51730d-21bf-4757-9a86-c2b8be482b47 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:55.255926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:55.260216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 431/882] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 9 18:30:55 2022] GET /identity/v3/endpoints?service_id=fc000368006949e0ae12d01e5be747fc&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:55.261143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea182098-74a0-4943-9eac-89332c6cb931 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:55.261426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea182098-74a0-4943-9eac-89332c6cb931 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:55.261764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea182098-74a0-4943-9eac-89332c6cb931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:55.279675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ea182098-74a0-4943-9eac-89332c6cb931 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:55.280857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ea182098-74a0-4943-9eac-89332c6cb931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:55.340058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a67fd-ed9d-46d9-923c-36a5bb002ca0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:55.366851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 432/883] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:55.366851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea182098-74a0-4943-9eac-89332c6cb931 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:55.378662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:55.386018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5f852-1e58-4854-a9ca-4d070e279f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:55.386603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5f852-1e58-4854-a9ca-4d070e279f3a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:55.386877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5f852-1e58-4854-a9ca-4d070e279f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:55.387144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5f852-1e58-4854-a9ca-4d070e279f3a None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:55.387923 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5f852-1e58-4854-a9ca-4d070e279f3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:55.388710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5f852-1e58-4854-a9ca-4d070e279f3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:55.390612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5f852-1e58-4854-a9ca-4d070e279f3a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:55.405573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 452/884] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:30:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:55.406539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5f852-1e58-4854-a9ca-4d070e279f3a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:55.485530 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea182098-74a0-4943-9eac-89332c6cb931 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:55.503707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:55.516137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5f852-1e58-4854-a9ca-4d070e279f3a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:55.521196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce43a09-ac1b-4eae-a73a-40c775150b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:55.522385 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce43a09-ac1b-4eae-a73a-40c775150b23 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:55.522666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce43a09-ac1b-4eae-a73a-40c775150b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:55.522930 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce43a09-ac1b-4eae-a73a-40c775150b23 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:55.523554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce43a09-ac1b-4eae-a73a-40c775150b23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:55.523858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce43a09-ac1b-4eae-a73a-40c775150b23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:55.525822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce43a09-ac1b-4eae-a73a-40c775150b23 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:55.530404 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 433/885] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 9 18:30:55 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:55.531159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce43a09-ac1b-4eae-a73a-40c775150b23 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:55.554491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fe304-993f-4890-b2e1-3d62716e3b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:55.555188 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fe304-993f-4890-b2e1-3d62716e3b10 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:55.555479 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fe304-993f-4890-b2e1-3d62716e3b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:55.555842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fe304-993f-4890-b2e1-3d62716e3b10 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:55.556507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fe304-993f-4890-b2e1-3d62716e3b10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:55.557979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fe304-993f-4890-b2e1-3d62716e3b10 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:55.578272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 453/886] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 9 18:30:55 2022] GET /identity/v3/services => generated 1011 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:30:55.579500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fe304-993f-4890-b2e1-3d62716e3b10 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:55.652121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:55.693392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce43a09-ac1b-4eae-a73a-40c775150b23 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:55.712789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fe304-993f-4890-b2e1-3d62716e3b10 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:55.750882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2e7ce-813c-4a5a-94af-298bd6e4baf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:55.751873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2e7ce-813c-4a5a-94af-298bd6e4baf7 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:55.752365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2e7ce-813c-4a5a-94af-298bd6e4baf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:55.752751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2e7ce-813c-4a5a-94af-298bd6e4baf7 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:55.753886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2e7ce-813c-4a5a-94af-298bd6e4baf7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:55.757073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2e7ce-813c-4a5a-94af-298bd6e4baf7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:55.774420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67fd09-95c6-44de-8164-fa893a7b99b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:55.774420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67fd09-95c6-44de-8164-fa893a7b99b7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:55.774691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67fd09-95c6-44de-8164-fa893a7b99b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:55.775010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67fd09-95c6-44de-8164-fa893a7b99b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:55.775857 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67fd09-95c6-44de-8164-fa893a7b99b7 None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=1255cb6e360244ba9d5cec5cc65a1f8a) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:55.776167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67fd09-95c6-44de-8164-fa893a7b99b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:55.777667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67fd09-95c6-44de-8164-fa893a7b99b7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:55.786685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 454/887] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:30:55 2022] GET /identity/v3/role_assignments?user.id=1255cb6e360244ba9d5cec5cc65a1f8a&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:55.863608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 434/888] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:30:55 2022] POST /identity/v3/services => generated 269 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:57.802782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e04b3-f18b-48cc-972b-9374d8b12499 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:57.803374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e04b3-f18b-48cc-972b-9374d8b12499 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:57.803454 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e04b3-f18b-48cc-972b-9374d8b12499 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:57.805348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 455/889] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:57.811135 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563ad52-974d-444f-a4fc-632d77306331 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:57.811713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563ad52-974d-444f-a4fc-632d77306331 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:57.812110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563ad52-974d-444f-a4fc-632d77306331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:57.835935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c563ad52-974d-444f-a4fc-632d77306331 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:57.837140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c563ad52-974d-444f-a4fc-632d77306331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:57.917868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 435/890] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:57.936560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e04b3-f18b-48cc-972b-9374d8b12499 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:57.956388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:58.007762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebabff-8a3a-4202-9f65-5b06120b179f None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.008116 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebabff-8a3a-4202-9f65-5b06120b179f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.008312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebabff-8a3a-4202-9f65-5b06120b179f None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.009970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 436/891] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:58.012727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd8d5a-0877-4906-9535-b2eb0e8b313a None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.012866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd8d5a-0877-4906-9535-b2eb0e8b313a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.013092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd8d5a-0877-4906-9535-b2eb0e8b313a None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.016938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 437/892] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:30:58.018740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eb454-14a3-4153-a3c4-12e54d7058a8 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.019028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eb454-14a3-4153-a3c4-12e54d7058a8 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.019305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eb454-14a3-4153-a3c4-12e54d7058a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.038114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a13eb454-14a3-4153-a3c4-12e54d7058a8 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:58.039819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a13eb454-14a3-4153-a3c4-12e54d7058a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:58.049013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e04b3-f18b-48cc-972b-9374d8b12499 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:58.085808 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38def72-897c-4227-803f-6829acea9b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:58.087289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38def72-897c-4227-803f-6829acea9b42 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.087289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38def72-897c-4227-803f-6829acea9b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.087289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38def72-897c-4227-803f-6829acea9b42 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.087766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38def72-897c-4227-803f-6829acea9b42 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:58.089128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38def72-897c-4227-803f-6829acea9b42 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-d38def72-897c-4227-803f-6829acea9b42 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:58.097779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:58.098825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:58.099704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 456/893] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:30:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:58.100669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbcf97-d80d-42c9-bee7-fe74a3e6e232 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.100669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbcf97-d80d-42c9-bee7-fe74a3e6e232 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.100669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbcf97-d80d-42c9-bee7-fe74a3e6e232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.101908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:58.113683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 438/894] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:58.115178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177abf87-19b5-4d11-85e6-2ada41123c5c None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.115980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177abf87-19b5-4d11-85e6-2ada41123c5c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.116264 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177abf87-19b5-4d11-85e6-2ada41123c5c None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.121584 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 439/895] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:30:58.121584 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177abf87-19b5-4d11-85e6-2ada41123c5c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:58.122835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bbbbcf97-d80d-42c9-bee7-fe74a3e6e232 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:58.124020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bbbbcf97-d80d-42c9-bee7-fe74a3e6e232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:58.192703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 457/896] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:58.194911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd425d-6c6e-489e-b2c9-9496971e59dc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.195202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd425d-6c6e-489e-b2c9-9496971e59dc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.195482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd425d-6c6e-489e-b2c9-9496971e59dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.216626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c4cd425d-6c6e-489e-b2c9-9496971e59dc None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:30:58.223997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c4cd425d-6c6e-489e-b2c9-9496971e59dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:30:58.223997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:58.231533 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177abf87-19b5-4d11-85e6-2ada41123c5c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:58.270274 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa93bd-6985-4251-9c93-f26cf3932aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:58.271325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa93bd-6985-4251-9c93-f26cf3932aeb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.271459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa93bd-6985-4251-9c93-f26cf3932aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.271736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa93bd-6985-4251-9c93-f26cf3932aeb None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.272378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa93bd-6985-4251-9c93-f26cf3932aeb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:58.272919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa93bd-6985-4251-9c93-f26cf3932aeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:58.274103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa93bd-6985-4251-9c93-f26cf3932aeb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:58.278874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 440/897] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:30:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:58.279664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa93bd-6985-4251-9c93-f26cf3932aeb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:58.292896 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 458/898] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:30:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:30:58.293786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd425d-6c6e-489e-b2c9-9496971e59dc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:58.352621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:58.394823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa93bd-6985-4251-9c93-f26cf3932aeb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:58.395068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd425d-6c6e-489e-b2c9-9496971e59dc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:58.429086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91a6e9-2630-4b0d-8230-265548b7db4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:58.429480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bb82e-2cf2-41b8-a804-51e7708d801c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:58.429759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a6e9-2630-4b0d-8230-265548b7db4f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.430042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a6e9-2630-4b0d-8230-265548b7db4f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.430144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bb82e-2cf2-41b8-a804-51e7708d801c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.430316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a6e9-2630-4b0d-8230-265548b7db4f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.430408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bb82e-2cf2-41b8-a804-51e7708d801c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.430749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bb82e-2cf2-41b8-a804-51e7708d801c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.431097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91a6e9-2630-4b0d-8230-265548b7db4f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:58.432530 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91a6e9-2630-4b0d-8230-265548b7db4f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:58.438472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-7c91a6e9-2630-4b0d-8230-265548b7db4f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:58.441206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:58.442186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 18:30:58.443084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:58.444934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bb82e-2cf2-41b8-a804-51e7708d801c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:30:58.445015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 441/899] 10.222.0.57 () {56 vars in 1132 bytes} [Tue Aug 9 18:30:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:58.445015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91a6e9-2630-4b0d-8230-265548b7db4f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-a91bb82e-2cf2-41b8-a804-51e7708d801c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:30:58.446685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:58.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 459/900] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Aug 9 18:30:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:58.448780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bb82e-2cf2-41b8-a804-51e7708d801c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:58.548699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bb82e-2cf2-41b8-a804-51e7708d801c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:58.551286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91a6e9-2630-4b0d-8230-265548b7db4f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:58.569090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:58.583604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa5657-27d1-45ae-880b-419f7c1b9570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:58.584340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa5657-27d1-45ae-880b-419f7c1b9570 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.584510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa5657-27d1-45ae-880b-419f7c1b9570 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.584777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa5657-27d1-45ae-880b-419f7c1b9570 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.585112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157fec3-1d8d-4a87-a739-8250c40a1202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:58.585414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa5657-27d1-45ae-880b-419f7c1b9570 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:58.585739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157fec3-1d8d-4a87-a739-8250c40a1202 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.586023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157fec3-1d8d-4a87-a739-8250c40a1202 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.586292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157fec3-1d8d-4a87-a739-8250c40a1202 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.586809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa5657-27d1-45ae-880b-419f7c1b9570 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:58.586918 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157fec3-1d8d-4a87-a739-8250c40a1202 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:58.587200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157fec3-1d8d-4a87-a739-8250c40a1202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:58.588538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157fec3-1d8d-4a87-a739-8250c40a1202 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:58.593356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 442/901] 10.222.0.57 () {56 vars in 1114 bytes} [Tue Aug 9 18:30:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:58.594148 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157fec3-1d8d-4a87-a739-8250c40a1202 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:58.665171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 460/902] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:30:58 2022] POST /identity/v3/services => generated 241 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:58.665948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa5657-27d1-45ae-880b-419f7c1b9570 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:58.695459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157fec3-1d8d-4a87-a739-8250c40a1202 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:58.738321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:58.738989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf224b11-730d-4ead-a892-d89fc1d12406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:58.739607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf224b11-730d-4ead-a892-d89fc1d12406 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.739890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf224b11-730d-4ead-a892-d89fc1d12406 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.740163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf224b11-730d-4ead-a892-d89fc1d12406 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.741446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf224b11-730d-4ead-a892-d89fc1d12406 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:58.741787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf224b11-730d-4ead-a892-d89fc1d12406 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:58.743199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf224b11-730d-4ead-a892-d89fc1d12406 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:58.748456 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 443/903] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 9 18:30:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:58.750652 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf224b11-730d-4ead-a892-d89fc1d12406 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:58.794711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa5657-27d1-45ae-880b-419f7c1b9570 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:58.800440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:58.837094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b787e-eb5f-486e-bcd5-ab09de7c9001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:58.837813 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b787e-eb5f-486e-bcd5-ab09de7c9001 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.838019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b787e-eb5f-486e-bcd5-ab09de7c9001 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.838287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b787e-eb5f-486e-bcd5-ab09de7c9001 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.838962 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b787e-eb5f-486e-bcd5-ab09de7c9001 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:58.839265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b787e-eb5f-486e-bcd5-ab09de7c9001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:58.840644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b787e-eb5f-486e-bcd5-ab09de7c9001 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:58.852287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 461/904] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 9 18:30:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:58.853147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b787e-eb5f-486e-bcd5-ab09de7c9001 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:58.869134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf224b11-730d-4ead-a892-d89fc1d12406 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:58.904046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20adc6-2b97-4aee-93b9-4b31045ec813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:58.904653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20adc6-2b97-4aee-93b9-4b31045ec813 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:58.904936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20adc6-2b97-4aee-93b9-4b31045ec813 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:58.905209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20adc6-2b97-4aee-93b9-4b31045ec813 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:58.905898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20adc6-2b97-4aee-93b9-4b31045ec813 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:58.907269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20adc6-2b97-4aee-93b9-4b31045ec813 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:58.963510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b787e-eb5f-486e-bcd5-ab09de7c9001 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:58.986549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:58.996276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 444/905] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:30:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:30:59.003583 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20adc6-2b97-4aee-93b9-4b31045ec813 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:59.006659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31cef4-ebd8-4729-8f2c-587266a46857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:59.007540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31cef4-ebd8-4729-8f2c-587266a46857 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:59.008042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31cef4-ebd8-4729-8f2c-587266a46857 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:59.008502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31cef4-ebd8-4729-8f2c-587266a46857 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:59.009324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31cef4-ebd8-4729-8f2c-587266a46857 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:59.009858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31cef4-ebd8-4729-8f2c-587266a46857 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:59.011526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31cef4-ebd8-4729-8f2c-587266a46857 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:59.019411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 462/906] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 9 18:30:58 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:59.025934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31cef4-ebd8-4729-8f2c-587266a46857 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:59.155857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:59.162813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20adc6-2b97-4aee-93b9-4b31045ec813 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:59.173919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31cef4-ebd8-4729-8f2c-587266a46857 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:59.212375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe04c3-3655-46ac-b083-b8d966b2686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:59.213018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe04c3-3655-46ac-b083-b8d966b2686d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:59.213300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe04c3-3655-46ac-b083-b8d966b2686d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:59.213657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe04c3-3655-46ac-b083-b8d966b2686d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:59.222531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a775b9-e7bd-4d15-8922-48f062110221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:59.223473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a775b9-e7bd-4d15-8922-48f062110221 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:59.223970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a775b9-e7bd-4d15-8922-48f062110221 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:59.224444 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a775b9-e7bd-4d15-8922-48f062110221 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:59.225945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a775b9-e7bd-4d15-8922-48f062110221 None admin] RBAC: Adding query filter params (interface=public, service_id=cb21e82e2d47417a9414b5a17ee05a3a, region_id=RegionOne) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:59.226070 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe04c3-3655-46ac-b083-b8d966b2686d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:59.226643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a775b9-e7bd-4d15-8922-48f062110221 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:59.227500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe04c3-3655-46ac-b083-b8d966b2686d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:59.228413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a775b9-e7bd-4d15-8922-48f062110221 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:59.239216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 463/907] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 9 18:30:59 2022] GET /identity/v3/endpoints?service_id=cb21e82e2d47417a9414b5a17ee05a3a&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-7fbe04c3-3655-46ac-b083-b8d966b2686d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:30:59.244180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:59.245144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:30:59.246603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:30:59.246603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:30:59.246603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:30:59.246603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 445/908] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:59.247823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a775b9-e7bd-4d15-8922-48f062110221 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:59.251678 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe04c3-3655-46ac-b083-b8d966b2686d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:59.313853 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:59.373619 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a775b9-e7bd-4d15-8922-48f062110221 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:59.394495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe04c3-3655-46ac-b083-b8d966b2686d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:59.410648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566912d-e1b3-460c-8fc1-b879b3e8c9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:59.411413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566912d-e1b3-460c-8fc1-b879b3e8c9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:59.411844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566912d-e1b3-460c-8fc1-b879b3e8c9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:59.412219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566912d-e1b3-460c-8fc1-b879b3e8c9e6 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:59.413787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566912d-e1b3-460c-8fc1-b879b3e8c9e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:59.415287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566912d-e1b3-460c-8fc1-b879b3e8c9e6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:59.422052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 464/909] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 9 18:30:59 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:30:59.438931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c73aac-78dd-477b-80e0-92ff9818812c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:59.439818 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c73aac-78dd-477b-80e0-92ff9818812c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:59.440245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c73aac-78dd-477b-80e0-92ff9818812c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:59.440692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c73aac-78dd-477b-80e0-92ff9818812c None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:59.441603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c73aac-78dd-477b-80e0-92ff9818812c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:59.442057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c73aac-78dd-477b-80e0-92ff9818812c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:59.443690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c73aac-78dd-477b-80e0-92ff9818812c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:59.459271 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 446/910] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:59.468388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566912d-e1b3-460c-8fc1-b879b3e8c9e6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:59.585704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566912d-e1b3-460c-8fc1-b879b3e8c9e6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:59.598753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:59.622987 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ca433-7c72-4d85-8059-32bb489b6519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:59.623959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ca433-7c72-4d85-8059-32bb489b6519 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:59.624511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ca433-7c72-4d85-8059-32bb489b6519 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:59.625126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ca433-7c72-4d85-8059-32bb489b6519 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:59.633231 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ca433-7c72-4d85-8059-32bb489b6519 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:59.634889 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ca433-7c72-4d85-8059-32bb489b6519 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-6c3ca433-7c72-4d85-8059-32bb489b6519 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:30:59.646480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:30:59.648201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:30:59.649351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:30:59.649351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:30:59.649351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:30:59.649351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:30:59.649351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:30:59.649351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:30:59.649351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:30:59.649351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:30:59.649351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 18:30:59.649351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:30:59.650688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 465/911] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 9 18:30:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:30:59.656296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c73aac-78dd-477b-80e0-92ff9818812c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:59.672207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:30:59.763823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c73aac-78dd-477b-80e0-92ff9818812c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:59.801871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b0c8a-7363-45d7-959b-e8ca22f50aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:59.802875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b0c8a-7363-45d7-959b-e8ca22f50aca None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:59.803450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b0c8a-7363-45d7-959b-e8ca22f50aca None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:59.804190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b0c8a-7363-45d7-959b-e8ca22f50aca None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:30:59.805149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b0c8a-7363-45d7-959b-e8ca22f50aca None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:30:59.805740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b0c8a-7363-45d7-959b-e8ca22f50aca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:30:59.807485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b0c8a-7363-45d7-959b-e8ca22f50aca None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:30:59.827988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 447/912] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Aug 9 18:30:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:30:59.837586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ca433-7c72-4d85-8059-32bb489b6519 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:30:59.946417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ca433-7c72-4d85-8059-32bb489b6519 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:30:59.984504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c44554-f800-401f-b180-9a8aabcffc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:30:59.985119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c44554-f800-401f-b180-9a8aabcffc26 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:30:59.985399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c44554-f800-401f-b180-9a8aabcffc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:30:59.985704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c44554-f800-401f-b180-9a8aabcffc26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:00.031237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:00.064898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c44554-f800-401f-b180-9a8aabcffc26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:00.066730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c44554-f800-401f-b180-9a8aabcffc26 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:00.177539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 466/913] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:30:59 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:31:00.184362 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b0c8a-7363-45d7-959b-e8ca22f50aca None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:00.285969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:00.310685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b0c8a-7363-45d7-959b-e8ca22f50aca None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:00.345938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e48c2-30ec-45dc-8b08-de47995732ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:00.346560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e48c2-30ec-45dc-8b08-de47995732ca None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:00.346834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e48c2-30ec-45dc-8b08-de47995732ca None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:00.347101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e48c2-30ec-45dc-8b08-de47995732ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:00.358447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e48c2-30ec-45dc-8b08-de47995732ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:00.359892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e48c2-30ec-45dc-8b08-de47995732ca None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-334e48c2-30ec-45dc-8b08-de47995732ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:00.374863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:00.376168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:31:00.377099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:00.377099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:00.377099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:00.377099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 448/914] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:00.382034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c44554-f800-401f-b180-9a8aabcffc26 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:00.461111 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:00.498281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c44554-f800-401f-b180-9a8aabcffc26 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:00.535035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174d9ce-449f-46d6-b228-ed124312e71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:00.535639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174d9ce-449f-46d6-b228-ed124312e71b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:00.535914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174d9ce-449f-46d6-b228-ed124312e71b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:00.536237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174d9ce-449f-46d6-b228-ed124312e71b None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:00.536883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174d9ce-449f-46d6-b228-ed124312e71b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:00.537188 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174d9ce-449f-46d6-b228-ed124312e71b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:00.539168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174d9ce-449f-46d6-b228-ed124312e71b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:00.551500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 467/915] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:00.557352 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e48c2-30ec-45dc-8b08-de47995732ca None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:00.667335 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e48c2-30ec-45dc-8b08-de47995732ca None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:00.687814 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:00.702170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c4940-a2c9-4607-ac1d-fec891967cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:00.702800 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c4940-a2c9-4607-ac1d-fec891967cab None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:00.703076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c4940-a2c9-4607-ac1d-fec891967cab None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:00.703345 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c4940-a2c9-4607-ac1d-fec891967cab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:00.710795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c4940-a2c9-4607-ac1d-fec891967cab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:00.712233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c4940-a2c9-4607-ac1d-fec891967cab None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-369c4940-a2c9-4607-ac1d-fec891967cab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:31:00.723868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:00.724919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:00.725843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:31:00.725843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:31:00.725843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:31:00.725843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:31:00.725843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:31:00.725843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:31:00.725843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:00.725843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:00.725843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 449/916] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:31:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:00.732764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174d9ce-449f-46d6-b228-ed124312e71b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:00.767404 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bce9f-4423-4d75-bfd8-28997bfea2f4 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:00.767752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bce9f-4423-4d75-bfd8-28997bfea2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:00.768058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bce9f-4423-4d75-bfd8-28997bfea2f4 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:00.769684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 450/917] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:31:00.775148 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfd098-a8e8-4dfc-85cf-69c86a234c30 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:00.775435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfd098-a8e8-4dfc-85cf-69c86a234c30 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:00.775711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfd098-a8e8-4dfc-85cf-69c86a234c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:00.794836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3cdfd098-a8e8-4dfc-85cf-69c86a234c30 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:00.795941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3cdfd098-a8e8-4dfc-85cf-69c86a234c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:00.860266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174d9ce-449f-46d6-b228-ed124312e71b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:00.889635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 451/918] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:31:00.903283 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:00.909333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473d9a6-2bc7-443b-9db0-70512e12adbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:00.909866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473d9a6-2bc7-443b-9db0-70512e12adbe None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:00.914949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473d9a6-2bc7-443b-9db0-70512e12adbe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:00.914949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473d9a6-2bc7-443b-9db0-70512e12adbe None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:00.914949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c473d9a6-2bc7-443b-9db0-70512e12adbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:00.915196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfd098-a8e8-4dfc-85cf-69c86a234c30 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:00.915278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c473d9a6-2bc7-443b-9db0-70512e12adbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:00.915523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c473d9a6-2bc7-443b-9db0-70512e12adbe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:00.930977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 468/919] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:31:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:00.936783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473d9a6-2bc7-443b-9db0-70512e12adbe None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:01.026626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfd098-a8e8-4dfc-85cf-69c86a234c30 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:01.039109 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:01.064982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0654dd-95c5-41b3-b9dc-4fe1b065f511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:01.066078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0654dd-95c5-41b3-b9dc-4fe1b065f511 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:01.066473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0654dd-95c5-41b3-b9dc-4fe1b065f511 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:01.066961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0654dd-95c5-41b3-b9dc-4fe1b065f511 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:01.067946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0654dd-95c5-41b3-b9dc-4fe1b065f511 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:01.070778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0654dd-95c5-41b3-b9dc-4fe1b065f511 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:01.076247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473d9a6-2bc7-443b-9db0-70512e12adbe None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-7a0654dd-95c5-41b3-b9dc-4fe1b065f511 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:01.082569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:01.085255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:01.086616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 452/920] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 9 18:31:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:01.090873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0654dd-95c5-41b3-b9dc-4fe1b065f511 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:01.115152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15374ea4-94e3-4667-9cf4-c47ae6659dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:01.116512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15374ea4-94e3-4667-9cf4-c47ae6659dcc None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:01.116793 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15374ea4-94e3-4667-9cf4-c47ae6659dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:01.117065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15374ea4-94e3-4667-9cf4-c47ae6659dcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:01.117826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15374ea4-94e3-4667-9cf4-c47ae6659dcc None admin] RBAC: Adding query filter params (role.id=039a8882198d4b17911651cf3558be04, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=1255cb6e360244ba9d5cec5cc65a1f8a) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:01.118133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15374ea4-94e3-4667-9cf4-c47ae6659dcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:01.119594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15374ea4-94e3-4667-9cf4-c47ae6659dcc None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:01.125674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 469/921] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:31:00 2022] GET /identity/v3/role_assignments?user.id=1255cb6e360244ba9d5cec5cc65a1f8a&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=039a8882198d4b17911651cf3558be04 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:01.184357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e2e94-9ba8-4b6a-85c9-1c82d36f4abe None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:01.184680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e2e94-9ba8-4b6a-85c9-1c82d36f4abe None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:01.184905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e2e94-9ba8-4b6a-85c9-1c82d36f4abe None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:01.186630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 470/922] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:31:01.192389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6039e54-4d2c-408f-a47a-bec1b9e1aa1e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:01.192686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6039e54-4d2c-408f-a47a-bec1b9e1aa1e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:01.193263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6039e54-4d2c-408f-a47a-bec1b9e1aa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:01.212944 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0654dd-95c5-41b3-b9dc-4fe1b065f511 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:01.219676 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b6039e54-4d2c-408f-a47a-bec1b9e1aa1e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:01.220906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b6039e54-4d2c-408f-a47a-bec1b9e1aa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:01.269613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2da2d-9180-4ccb-9f21-f6a02d48ab40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:01.270933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2da2d-9180-4ccb-9f21-f6a02d48ab40 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:01.270933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2da2d-9180-4ccb-9f21-f6a02d48ab40 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:01.270933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2da2d-9180-4ccb-9f21-f6a02d48ab40 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:01.272198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2da2d-9180-4ccb-9f21-f6a02d48ab40 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:01.272198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2da2d-9180-4ccb-9f21-f6a02d48ab40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:01.277549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2da2d-9180-4ccb-9f21-f6a02d48ab40 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:01.282660 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 453/923] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 9 18:31:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:01.288368 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2da2d-9180-4ccb-9f21-f6a02d48ab40 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:01.294023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:01.317593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 471/924] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:01.340987 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6039e54-4d2c-408f-a47a-bec1b9e1aa1e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:01.342126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:01.412051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2da2d-9180-4ccb-9f21-f6a02d48ab40 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:01.449825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9091dc-8c1e-47ce-a911-4edadc7f3c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:01.450473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9091dc-8c1e-47ce-a911-4edadc7f3c75 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:01.450748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9091dc-8c1e-47ce-a911-4edadc7f3c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:01.451061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9091dc-8c1e-47ce-a911-4edadc7f3c75 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:01.451698 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9091dc-8c1e-47ce-a911-4edadc7f3c75 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:01.451997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9091dc-8c1e-47ce-a911-4edadc7f3c75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:01.453365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9091dc-8c1e-47ce-a911-4edadc7f3c75 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:01.458895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 454/925] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 9 18:31:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:01.464515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9091dc-8c1e-47ce-a911-4edadc7f3c75 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:01.489770 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6039e54-4d2c-408f-a47a-bec1b9e1aa1e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:01.529921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcff38-774b-43fc-8a20-d766d0139691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:01.530674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcff38-774b-43fc-8a20-d766d0139691 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:01.530957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcff38-774b-43fc-8a20-d766d0139691 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:01.531229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcff38-774b-43fc-8a20-d766d0139691 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:01.531987 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcff38-774b-43fc-8a20-d766d0139691 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:01.533539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcff38-774b-43fc-8a20-d766d0139691 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:01.536473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: WARNING keystone.server.flask.common [None req-9cbcff38-774b-43fc-8a20-d766d0139691 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 09 18:31:01.578515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9091dc-8c1e-47ce-a911-4edadc7f3c75 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:01.578515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:01.604407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d811c1-05eb-45bf-aff5-12a392a823da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:01.605006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d811c1-05eb-45bf-aff5-12a392a823da None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:01.605313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d811c1-05eb-45bf-aff5-12a392a823da None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:01.605616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d811c1-05eb-45bf-aff5-12a392a823da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:01.606326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d811c1-05eb-45bf-aff5-12a392a823da None admin] RBAC: Adding query filter params (interface=public, service_id=3288e08e34304e1eb74dbd2ee97f1a49, region_id=RegionOne) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:01.606629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d811c1-05eb-45bf-aff5-12a392a823da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:01.606802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.sql.core [None req-9cbcff38-774b-43fc-8a20-d766d0139691 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 18:31:01.606802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 18:31:01.606802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [parameters: {'id': 'bf6d5191cbc44a32807117e732d81af9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 18:31:01.606802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80446) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 18:31:01.608046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d811c1-05eb-45bf-aff5-12a392a823da None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:01.614028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 455/926] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 9 18:31:01 2022] GET /identity/v3/endpoints?service_id=3288e08e34304e1eb74dbd2ee97f1a49&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:01.623027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d811c1-05eb-45bf-aff5-12a392a823da None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-9cbcff38-774b-43fc-8a20-d766d0139691 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bf6d5191cbc44a32807117e732d81af9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bf6d5191cbc44a32807117e732d81af9. Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application conn.query(q) Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 18:31:01.630187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result.read() Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 09 18:31:01.631185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application next(self.gen) Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application yield resource Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application next(self.gen) Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise self.value Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application session.commit() Aug 09 18:31:01.632407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.session.flush() Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application compat.raise_( Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 18:31:01.634327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 18:31:01.635973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application conn.query(q) Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result.read() Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 18:31:01.637340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [parameters: {'id': 'bf6d5191cbc44a32807117e732d81af9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:01.642589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:01.643388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:01.644412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:01.644412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 09 18:31:01.644412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 09 18:31:01.644412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:01.644412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:01.644412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 09 18:31:01.644412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 09 18:31:01.644412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bf6d5191cbc44a32807117e732d81af9. Aug 09 18:31:01.644412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:01.644412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 472/927] 10.222.0.57 () {60 vars in 1140 bytes} [Tue Aug 9 18:31:01 2022] POST /identity/v3/projects => generated 262 bytes in 294 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:01.646845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcff38-774b-43fc-8a20-d766d0139691 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:01.700035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:01.754799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d811c1-05eb-45bf-aff5-12a392a823da None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:01.764880 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcff38-774b-43fc-8a20-d766d0139691 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:01.805187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc583f00-6a2c-4e50-87a9-d01fc5579e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:01.806336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc583f00-6a2c-4e50-87a9-d01fc5579e83 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:01.806885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc583f00-6a2c-4e50-87a9-d01fc5579e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:01.806885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc583f00-6a2c-4e50-87a9-d01fc5579e83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:01.810447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb9405-7626-4df8-aa3b-33c30fec550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:01.811194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb9405-7626-4df8-aa3b-33c30fec550d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:01.811684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb9405-7626-4df8-aa3b-33c30fec550d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:01.811684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb9405-7626-4df8-aa3b-33c30fec550d None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:01.812369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb9405-7626-4df8-aa3b-33c30fec550d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:01.814007 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb9405-7626-4df8-aa3b-33c30fec550d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:01.815403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc583f00-6a2c-4e50-87a9-d01fc5579e83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:01.817188 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc583f00-6a2c-4e50-87a9-d01fc5579e83 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:01.820908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 456/928] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 9 18:31:01 2022] GET /identity/v3/services => generated 1497 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:31:01.822397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65f283-d59a-4ff6-9100-89a742873e7e None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:01.822582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65f283-d59a-4ff6-9100-89a742873e7e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:01.822737 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65f283-d59a-4ff6-9100-89a742873e7e None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:01.828265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 457/929] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:01.829943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b5a0f-6dba-45f0-9185-b3ed5fb6177e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:01.830492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b5a0f-6dba-45f0-9185-b3ed5fb6177e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:01.830492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b5a0f-6dba-45f0-9185-b3ed5fb6177e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-dc583f00-6a2c-4e50-87a9-d01fc5579e83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:31:01.846448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:01.847415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:01.848316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:31:01.848316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:31:01.848316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:31:01.848316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:31:01.848316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:31:01.848316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:31:01.848316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:01.848316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:01.850704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 473/930] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:01.854125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc583f00-6a2c-4e50-87a9-d01fc5579e83 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:01.866153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-868b5a0f-6dba-45f0-9185-b3ed5fb6177e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:01.867466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-868b5a0f-6dba-45f0-9185-b3ed5fb6177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:01.912861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:01.969373 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 458/931] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:01.980812 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:01.984550 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc583f00-6a2c-4e50-87a9-d01fc5579e83 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:01.986822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b5a0f-6dba-45f0-9185-b3ed5fb6177e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:02.034276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515c1a3-77d5-4a9f-9cb3-14f65cfe33ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:02.034888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515c1a3-77d5-4a9f-9cb3-14f65cfe33ca None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:02.035180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515c1a3-77d5-4a9f-9cb3-14f65cfe33ca None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:02.035448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515c1a3-77d5-4a9f-9cb3-14f65cfe33ca None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:02.036104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515c1a3-77d5-4a9f-9cb3-14f65cfe33ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:02.036405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515c1a3-77d5-4a9f-9cb3-14f65cfe33ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:02.038100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515c1a3-77d5-4a9f-9cb3-14f65cfe33ca None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:02.074342 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 474/932] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:31:01 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:02.119994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b5a0f-6dba-45f0-9185-b3ed5fb6177e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:02.159995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9325f9-b540-44f7-a180-5d35bd4b4b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:02.161062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9325f9-b540-44f7-a180-5d35bd4b4b0f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:02.161622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9325f9-b540-44f7-a180-5d35bd4b4b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:02.162120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9325f9-b540-44f7-a180-5d35bd4b4b0f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:02.163478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9325f9-b540-44f7-a180-5d35bd4b4b0f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:02.165094 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9325f9-b540-44f7-a180-5d35bd4b4b0f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-0b9325f9-b540-44f7-a180-5d35bd4b4b0f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:02.175195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:02.176195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 18:31:02.177091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:02.181041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 459/933] 10.222.0.57 () {56 vars in 1132 bytes} [Tue Aug 9 18:31:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:02.186995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515c1a3-77d5-4a9f-9cb3-14f65cfe33ca None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:02.316359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515c1a3-77d5-4a9f-9cb3-14f65cfe33ca None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:02.356673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4a2d7-b5c9-4480-aea3-c7a66ece9818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:02.357285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4a2d7-b5c9-4480-aea3-c7a66ece9818 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:02.357593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4a2d7-b5c9-4480-aea3-c7a66ece9818 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:02.357867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4a2d7-b5c9-4480-aea3-c7a66ece9818 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:02.358517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4a2d7-b5c9-4480-aea3-c7a66ece9818 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:02.358821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4a2d7-b5c9-4480-aea3-c7a66ece9818 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:02.360213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4a2d7-b5c9-4480-aea3-c7a66ece9818 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:02.367898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 475/934] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 9 18:31:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:02.373773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9325f9-b540-44f7-a180-5d35bd4b4b0f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:02.413923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:02.495685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9325f9-b540-44f7-a180-5d35bd4b4b0f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:02.530625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ac80e-178c-43e6-88ec-da3cce28f0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:02.531315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ac80e-178c-43e6-88ec-da3cce28f0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:02.531597 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ac80e-178c-43e6-88ec-da3cce28f0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:02.531870 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ac80e-178c-43e6-88ec-da3cce28f0b9 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:02.532502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ac80e-178c-43e6-88ec-da3cce28f0b9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:02.532807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ac80e-178c-43e6-88ec-da3cce28f0b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:02.534198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ac80e-178c-43e6-88ec-da3cce28f0b9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:02.540135 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 460/935] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 9 18:31:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:02.545762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4a2d7-b5c9-4480-aea3-c7a66ece9818 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:02.576209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:02.655730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4a2d7-b5c9-4480-aea3-c7a66ece9818 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:02.692741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee4adf-9480-49cb-9a16-54ec1582ee36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:02.693801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee4adf-9480-49cb-9a16-54ec1582ee36 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:02.694295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee4adf-9480-49cb-9a16-54ec1582ee36 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:02.694771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee4adf-9480-49cb-9a16-54ec1582ee36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:02.695719 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee4adf-9480-49cb-9a16-54ec1582ee36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:02.697346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee4adf-9480-49cb-9a16-54ec1582ee36 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:02.784876 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 476/936] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:31:02 2022] POST /identity/v3/endpoints => generated 347 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:31:03.243449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ff89-f3f6-414d-846b-851d788d3a02 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:03.243872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ff89-f3f6-414d-846b-851d788d3a02 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:03.243976 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ff89-f3f6-414d-846b-851d788d3a02 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:03.245643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 461/937] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:03.250875 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf66f90-6cd0-4bf3-b8c6-4cb649c36b28 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:03.251335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf66f90-6cd0-4bf3-b8c6-4cb649c36b28 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:03.251413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf66f90-6cd0-4bf3-b8c6-4cb649c36b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:03.273090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8cf66f90-6cd0-4bf3-b8c6-4cb649c36b28 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:03.274664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8cf66f90-6cd0-4bf3-b8c6-4cb649c36b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:03.351291 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 477/938] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:03.374202 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9ff89-f3f6-414d-846b-851d788d3a02 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:03.466010 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:03.479485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9ff89-f3f6-414d-846b-851d788d3a02 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:03.515221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbba97-0f80-4200-85f5-6860113708f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:03.515827 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbba97-0f80-4200-85f5-6860113708f7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:03.516113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbba97-0f80-4200-85f5-6860113708f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:03.516385 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbba97-0f80-4200-85f5-6860113708f7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:03.526750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbba97-0f80-4200-85f5-6860113708f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-10fbba97-0f80-4200-85f5-6860113708f7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:31:03.531620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:03.532555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:03.533810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 462/939] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Aug 9 18:31:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:03.539121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf66f90-6cd0-4bf3-b8c6-4cb649c36b28 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:03.549902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:03.646137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf66f90-6cd0-4bf3-b8c6-4cb649c36b28 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:03.682146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7795f03-640d-448e-b64f-6e590ee29c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:03.682999 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7795f03-640d-448e-b64f-6e590ee29c55 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:03.683449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7795f03-640d-448e-b64f-6e590ee29c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:03.683857 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7795f03-640d-448e-b64f-6e590ee29c55 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:03.684650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7795f03-640d-448e-b64f-6e590ee29c55 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:03.685103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7795f03-640d-448e-b64f-6e590ee29c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:03.686775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7795f03-640d-448e-b64f-6e590ee29c55 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:03.695473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 478/940] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 9 18:31:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:03.702457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbba97-0f80-4200-85f5-6860113708f7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:03.810252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbba97-0f80-4200-85f5-6860113708f7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:03.845581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99961af9-1816-4cd2-8482-5ee21fee0f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:03.846960 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961af9-1816-4cd2-8482-5ee21fee0f3c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:03.847422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961af9-1816-4cd2-8482-5ee21fee0f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:03.847838 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961af9-1816-4cd2-8482-5ee21fee0f3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:03.859352 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99961af9-1816-4cd2-8482-5ee21fee0f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:03.861019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99961af9-1816-4cd2-8482-5ee21fee0f3c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:03.871152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-99961af9-1816-4cd2-8482-5ee21fee0f3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:03.878998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:03.880089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:31:03.881204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:03.881204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:03.881204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:03.882931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 463/941] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:03.890649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7795f03-640d-448e-b64f-6e590ee29c55 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:03.960778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:03.997081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7795f03-640d-448e-b64f-6e590ee29c55 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:04.033933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c88ed-0297-4397-ae3f-6b946e6fde75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:04.034541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c88ed-0297-4397-ae3f-6b946e6fde75 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.034817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c88ed-0297-4397-ae3f-6b946e6fde75 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.035086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c88ed-0297-4397-ae3f-6b946e6fde75 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.035728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c88ed-0297-4397-ae3f-6b946e6fde75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:04.036027 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c88ed-0297-4397-ae3f-6b946e6fde75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:04.037961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c88ed-0297-4397-ae3f-6b946e6fde75 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:04.050919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 479/942] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:04.058657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99961af9-1816-4cd2-8482-5ee21fee0f3c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:04.076419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2dc79-5159-4054-a24b-aaf0ba9b9974 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.076690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2dc79-5159-4054-a24b-aaf0ba9b9974 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.076905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2dc79-5159-4054-a24b-aaf0ba9b9974 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.078565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 480/943] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:31:04.083636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e808b-0619-45e0-9048-3e071f1e0bb3 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.084451 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e808b-0619-45e0-9048-3e071f1e0bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.084733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e808b-0619-45e0-9048-3e071f1e0bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.104129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6d1e808b-0619-45e0-9048-3e071f1e0bb3 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:04.105337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6d1e808b-0619-45e0-9048-3e071f1e0bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:04.172680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99961af9-1816-4cd2-8482-5ee21fee0f3c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:04.186203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 481/944] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:31:04.208172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e808b-0619-45e0-9048-3e071f1e0bb3 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:04.213485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385aff85-7c57-48fd-95f2-5b852dd9d367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:04.214194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385aff85-7c57-48fd-95f2-5b852dd9d367 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.214472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385aff85-7c57-48fd-95f2-5b852dd9d367 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.214742 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385aff85-7c57-48fd-95f2-5b852dd9d367 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.223133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385aff85-7c57-48fd-95f2-5b852dd9d367 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:04.225315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385aff85-7c57-48fd-95f2-5b852dd9d367 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-385aff85-7c57-48fd-95f2-5b852dd9d367 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:31:04.237549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:04.238647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:04.239916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:04.239916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:04.239916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:31:04.239916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:31:04.239916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:31:04.239916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:31:04.239916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:31:04.239916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:31:04.239916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 18:31:04.239916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:04.239916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 464/945] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 9 18:31:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:04.244527 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385aff85-7c57-48fd-95f2-5b852dd9d367 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:04.275361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:04.311512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:04.321298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e808b-0619-45e0-9048-3e071f1e0bb3 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:04.360714 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385aff85-7c57-48fd-95f2-5b852dd9d367 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:04.367586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1522e3-2899-4a19-969e-c7f015744a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:04.368378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1522e3-2899-4a19-969e-c7f015744a5e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.368774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1522e3-2899-4a19-969e-c7f015744a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.369410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1522e3-2899-4a19-969e-c7f015744a5e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.369950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1522e3-2899-4a19-969e-c7f015744a5e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:04.371577 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1522e3-2899-4a19-969e-c7f015744a5e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-9a1522e3-2899-4a19-969e-c7f015744a5e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:04.381078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:04.382183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:31:04.383042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:04.387740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 482/946] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 9 18:31:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:04.388666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1522e3-2899-4a19-969e-c7f015744a5e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:04.402302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a5a8a-d2d9-4d5a-8e37-6698f53b8deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:04.403775 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a5a8a-d2d9-4d5a-8e37-6698f53b8deb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.404434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a5a8a-d2d9-4d5a-8e37-6698f53b8deb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.404911 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a5a8a-d2d9-4d5a-8e37-6698f53b8deb None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.406104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a5a8a-d2d9-4d5a-8e37-6698f53b8deb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:04.406636 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a5a8a-d2d9-4d5a-8e37-6698f53b8deb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:04.408425 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a5a8a-d2d9-4d5a-8e37-6698f53b8deb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:04.433759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 465/947] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Aug 9 18:31:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:04.434924 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a5a8a-d2d9-4d5a-8e37-6698f53b8deb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:04.511890 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1522e3-2899-4a19-969e-c7f015744a5e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:04.537083 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:04.549510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7dacc-3e29-4d8f-87bc-1b752f946a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:04.550239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7dacc-3e29-4d8f-87bc-1b752f946a19 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.550526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7dacc-3e29-4d8f-87bc-1b752f946a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.550800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7dacc-3e29-4d8f-87bc-1b752f946a19 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.551441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7dacc-3e29-4d8f-87bc-1b752f946a19 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:04.557781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7dacc-3e29-4d8f-87bc-1b752f946a19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:04.557895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a5a8a-d2d9-4d5a-8e37-6698f53b8deb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:04.557991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7dacc-3e29-4d8f-87bc-1b752f946a19 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:04.563613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 483/948] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 9 18:31:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:04.564806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7dacc-3e29-4d8f-87bc-1b752f946a19 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:04.574314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:04.595024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d28ac-76dc-42e6-bc70-6531be6ac8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:04.595805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d28ac-76dc-42e6-bc70-6531be6ac8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.596185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d28ac-76dc-42e6-bc70-6531be6ac8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.596556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d28ac-76dc-42e6-bc70-6531be6ac8d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.609089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d28ac-76dc-42e6-bc70-6531be6ac8d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:04.610785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d28ac-76dc-42e6-bc70-6531be6ac8d3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-998d28ac-76dc-42e6-bc70-6531be6ac8d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:04.626862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:04.627923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:31:04.628791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:04.628791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:04.628791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:04.633718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 466/949] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:04.634494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d28ac-76dc-42e6-bc70-6531be6ac8d3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:04.681388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7dacc-3e29-4d8f-87bc-1b752f946a19 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:04.717744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddf2ab-fd85-4a4a-ac43-04d1fc851b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:04.718472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddf2ab-fd85-4a4a-ac43-04d1fc851b70 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.718758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddf2ab-fd85-4a4a-ac43-04d1fc851b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.719028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddf2ab-fd85-4a4a-ac43-04d1fc851b70 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.719868 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddf2ab-fd85-4a4a-ac43-04d1fc851b70 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:04.719986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddf2ab-fd85-4a4a-ac43-04d1fc851b70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:04.721325 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddf2ab-fd85-4a4a-ac43-04d1fc851b70 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:04.730151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 484/950] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 9 18:31:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:04.730151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8bafb-454e-4daf-85d0-2536938bc9b1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.730151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8bafb-454e-4daf-85d0-2536938bc9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.730151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8bafb-454e-4daf-85d0-2536938bc9b1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.731316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 485/951] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:04.736722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8bafb-454e-4daf-85d0-2536938bc9b1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:04.754178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d28ac-76dc-42e6-bc70-6531be6ac8d3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:04.793464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3f482-8af0-4125-841b-78cf7f18d995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:04.794098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3f482-8af0-4125-841b-78cf7f18d995 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.794379 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3f482-8af0-4125-841b-78cf7f18d995 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.794635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3f482-8af0-4125-841b-78cf7f18d995 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.795289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3f482-8af0-4125-841b-78cf7f18d995 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:04.795593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3f482-8af0-4125-841b-78cf7f18d995 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:04.796983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3f482-8af0-4125-841b-78cf7f18d995 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:04.809734 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:04.814839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 467/952] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:04.814839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45138a8b-1ad9-4f3d-83c8-9b17c177f827 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.814839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45138a8b-1ad9-4f3d-83c8-9b17c177f827 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.814839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45138a8b-1ad9-4f3d-83c8-9b17c177f827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.830220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-45138a8b-1ad9-4f3d-83c8-9b17c177f827 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:04.838878 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-45138a8b-1ad9-4f3d-83c8-9b17c177f827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:04.855639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:04.871287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8bafb-454e-4daf-85d0-2536938bc9b1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:04.907927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e9fb5-b477-42b6-952c-9fcf4019799b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:04.908541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e9fb5-b477-42b6-952c-9fcf4019799b None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:04.908816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e9fb5-b477-42b6-952c-9fcf4019799b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:04.909093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e9fb5-b477-42b6-952c-9fcf4019799b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:04.911007 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 468/953] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:04.911123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e9fb5-b477-42b6-952c-9fcf4019799b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:04.911199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45138a8b-1ad9-4f3d-83c8-9b17c177f827 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:04.911408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e9fb5-b477-42b6-952c-9fcf4019799b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:05.001356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 486/954] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:31:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:31:05.002232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e9fb5-b477-42b6-952c-9fcf4019799b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:05.026822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45138a8b-1ad9-4f3d-83c8-9b17c177f827 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:05.090846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05273fb1-f83d-444e-9f5e-f0f5e1314b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:05.091501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05273fb1-f83d-444e-9f5e-f0f5e1314b59 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:05.091793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05273fb1-f83d-444e-9f5e-f0f5e1314b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:05.092059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05273fb1-f83d-444e-9f5e-f0f5e1314b59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:05.101080 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05273fb1-f83d-444e-9f5e-f0f5e1314b59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:05.102443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05273fb1-f83d-444e-9f5e-f0f5e1314b59 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-05273fb1-f83d-444e-9f5e-f0f5e1314b59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:31:05.123344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:05.124347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:05.125735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:31:05.125735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:31:05.125735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:31:05.125735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:31:05.125735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:31:05.125735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:31:05.125735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:05.125735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:05.125735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 469/955] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:31:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:05.127423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bbd78-c89a-41f5-a977-48e3218833d7 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:05.127902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bbd78-c89a-41f5-a977-48e3218833d7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:05.128392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bbd78-c89a-41f5-a977-48e3218833d7 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:05.130827 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 470/956] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:05.132684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bbd78-c89a-41f5-a977-48e3218833d7 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:05.138898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e9fb5-b477-42b6-952c-9fcf4019799b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:05.149040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:05.160886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:05.203871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f64f4-8a5f-4df0-9990-32d51f3ce9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:05.204905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f64f4-8a5f-4df0-9990-32d51f3ce9c7 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:05.205378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f64f4-8a5f-4df0-9990-32d51f3ce9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:05.205873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f64f4-8a5f-4df0-9990-32d51f3ce9c7 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:05.207034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f64f4-8a5f-4df0-9990-32d51f3ce9c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:05.210363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f64f4-8a5f-4df0-9990-32d51f3ce9c7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:05.266612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.sql.core [None req-984f64f4-8a5f-4df0-9990-32d51f3ce9c7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 18:31:05.266612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 18:31:05.266612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [parameters: {'user_id': 'e5dd0ece90e648d3b8c5f77b97bc7f6d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 18:31:05.266612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80446) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 18:31:05.272060 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bbd78-c89a-41f5-a977-48e3218833d7 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-984f64f4-8a5f-4df0-9990-32d51f3ce9c7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application conn.query(q) Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result.read() Aug 09 18:31:05.275336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 18:31:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application next(self.gen) Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application yield resource Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application next(self.gen) Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise self.value Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application session.commit() Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 18:31:05.285017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.session.flush() Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application compat.raise_( Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 18:31:05.287971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 18:31:05.291322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application conn.query(q) Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application result.read() Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 18:31:05.295686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e5dd0ece90e648d3b8c5f77b97bc7f6d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:05.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 09 18:31:05.305596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 09 18:31:05.309403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 09 18:31:05.309403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 09 18:31:05.309403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 18:31:05.309403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 18:31:05.309403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 18:31:05.309403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:05.309403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 487/957] 10.222.0.57 () {60 vars in 1131 bytes} [Tue Aug 9 18:31:05 2022] POST /identity/v3/users => generated 160 bytes in 275 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:05.309403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d8376-5a03-47b0-8489-2adde9174a40 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:05.309403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d8376-5a03-47b0-8489-2adde9174a40 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:05.309403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d8376-5a03-47b0-8489-2adde9174a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:05.311317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a92d8376-5a03-47b0-8489-2adde9174a40 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:05.311513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a92d8376-5a03-47b0-8489-2adde9174a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:05.338994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426adaa-15a3-4646-b3dc-3151950cca80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:05.339807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426adaa-15a3-4646-b3dc-3151950cca80 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:05.340093 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426adaa-15a3-4646-b3dc-3151950cca80 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:05.340365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426adaa-15a3-4646-b3dc-3151950cca80 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:05.341335 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426adaa-15a3-4646-b3dc-3151950cca80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:05.341730 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426adaa-15a3-4646-b3dc-3151950cca80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:05.343203 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426adaa-15a3-4646-b3dc-3151950cca80 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:05.360575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 471/958] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:31:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:05.361434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426adaa-15a3-4646-b3dc-3151950cca80 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:05.394659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 488/959] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:05.395798 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d8376-5a03-47b0-8489-2adde9174a40 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:05.484512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:05.510884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426adaa-15a3-4646-b3dc-3151950cca80 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:05.526285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:05.531324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d8376-5a03-47b0-8489-2adde9174a40 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:05.554917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74d2b5-3d6c-4db1-8e08-0a7723dfb616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:05.555549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74d2b5-3d6c-4db1-8e08-0a7723dfb616 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:05.555828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74d2b5-3d6c-4db1-8e08-0a7723dfb616 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:05.556103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74d2b5-3d6c-4db1-8e08-0a7723dfb616 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:05.569447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74d2b5-3d6c-4db1-8e08-0a7723dfb616 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:05.570452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005eb085-9630-4112-a4eb-e038de1f35f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:05.575766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005eb085-9630-4112-a4eb-e038de1f35f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:05.575766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005eb085-9630-4112-a4eb-e038de1f35f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:05.575766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005eb085-9630-4112-a4eb-e038de1f35f3 None admin] PATH_INFO: `/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/1255cb6e360244ba9d5cec5cc65a1f8a/roles/039a8882198d4b17911651cf3558be04` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:05.576627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74d2b5-3d6c-4db1-8e08-0a7723dfb616 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:05.576714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005eb085-9630-4112-a4eb-e038de1f35f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=abcccf694e7b4f66a7372e0ede416caf, user_id=1255cb6e360244ba9d5cec5cc65a1f8a, role_id=039a8882198d4b17911651cf3558be04)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:05.577253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005eb085-9630-4112-a4eb-e038de1f35f3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-ef74d2b5-3d6c-4db1-8e08-0a7723dfb616 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:31:05.593278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:05.594466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:05.595415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:05.595415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:05.595415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:31:05.595415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:31:05.595415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:31:05.595415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:31:05.595415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:31:05.595415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:31:05.595415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 18:31:05.595415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:05.596458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 472/960] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:31:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:05.596912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74d2b5-3d6c-4db1-8e08-0a7723dfb616 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:05.681896 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 489/961] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:31:05 2022] PUT /identity/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/1255cb6e360244ba9d5cec5cc65a1f8a/roles/039a8882198d4b17911651cf3558be04 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:31:05.682865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005eb085-9630-4112-a4eb-e038de1f35f3 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:05.743375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74d2b5-3d6c-4db1-8e08-0a7723dfb616 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:05.795653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5513d81-ddb3-4925-a0f8-ca64282c7955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:05.796879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5513d81-ddb3-4925-a0f8-ca64282c7955 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:05.797421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5513d81-ddb3-4925-a0f8-ca64282c7955 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:05.797959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5513d81-ddb3-4925-a0f8-ca64282c7955 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:05.798793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5513d81-ddb3-4925-a0f8-ca64282c7955 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:05.800523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5513d81-ddb3-4925-a0f8-ca64282c7955 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:05.810086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-f5513d81-ddb3-4925-a0f8-ca64282c7955 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:05.820529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:05.824015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:05.824877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 473/962] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:31:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:05.835149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5513d81-ddb3-4925-a0f8-ca64282c7955 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:05.846733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005eb085-9630-4112-a4eb-e038de1f35f3 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:05.901393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0af2e-11ba-4da7-86ba-284a065497b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:05.902960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0af2e-11ba-4da7-86ba-284a065497b5 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:05.903485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0af2e-11ba-4da7-86ba-284a065497b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:05.904159 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0af2e-11ba-4da7-86ba-284a065497b5 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:05.905275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0af2e-11ba-4da7-86ba-284a065497b5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:05.905851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0af2e-11ba-4da7-86ba-284a065497b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:05.908118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0af2e-11ba-4da7-86ba-284a065497b5 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:05.964892 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 490/963] 10.222.0.57 () {56 vars in 1101 bytes} [Tue Aug 9 18:31:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:05.972579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5513d81-ddb3-4925-a0f8-ca64282c7955 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:06.013177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166f6e3-2216-45b4-b664-a2bbe5db1357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:06.013834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166f6e3-2216-45b4-b664-a2bbe5db1357 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:06.014116 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166f6e3-2216-45b4-b664-a2bbe5db1357 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:06.014391 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166f6e3-2216-45b4-b664-a2bbe5db1357 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:06.015043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166f6e3-2216-45b4-b664-a2bbe5db1357 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:06.015350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166f6e3-2216-45b4-b664-a2bbe5db1357 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:06.016751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166f6e3-2216-45b4-b664-a2bbe5db1357 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:06.023497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 474/964] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:31:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:06.030706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0af2e-11ba-4da7-86ba-284a065497b5 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:06.051823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:06.154399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0af2e-11ba-4da7-86ba-284a065497b5 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:06.190959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21a095-f9f2-499a-bd37-5196c3366342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:06.191578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21a095-f9f2-499a-bd37-5196c3366342 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:06.191855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21a095-f9f2-499a-bd37-5196c3366342 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:06.192119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21a095-f9f2-499a-bd37-5196c3366342 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:06.192756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21a095-f9f2-499a-bd37-5196c3366342 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:06.193052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21a095-f9f2-499a-bd37-5196c3366342 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:06.195038 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21a095-f9f2-499a-bd37-5196c3366342 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:06.200561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 491/965] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:31:06 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:07.137053 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35031a-f048-40f5-90b7-b43f8583864c None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:07.137626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35031a-f048-40f5-90b7-b43f8583864c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:07.137626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35031a-f048-40f5-90b7-b43f8583864c None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:07.139106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 475/966] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:07.144250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40448a-9a27-401f-a38a-44ba3ef8a527 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:07.144636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40448a-9a27-401f-a38a-44ba3ef8a527 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:07.144795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40448a-9a27-401f-a38a-44ba3ef8a527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:07.163441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1e40448a-9a27-401f-a38a-44ba3ef8a527 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:07.164605 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1e40448a-9a27-401f-a38a-44ba3ef8a527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:07.234501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 492/967] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:07.256300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35031a-f048-40f5-90b7-b43f8583864c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:07.334884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:07.375549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35031a-f048-40f5-90b7-b43f8583864c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:07.410065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa07756-be23-4fac-a89a-44a491e41938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:07.410620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa07756-be23-4fac-a89a-44a491e41938 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:07.410951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa07756-be23-4fac-a89a-44a491e41938 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:07.411223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa07756-be23-4fac-a89a-44a491e41938 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:07.411861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa07756-be23-4fac-a89a-44a491e41938 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:07.413802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa07756-be23-4fac-a89a-44a491e41938 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-2fa07756-be23-4fac-a89a-44a491e41938 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:07.422569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:07.429683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:07.430687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 476/968] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 9 18:31:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:07.432827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40448a-9a27-401f-a38a-44ba3ef8a527 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:07.445792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:07.539258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40448a-9a27-401f-a38a-44ba3ef8a527 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:07.574239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf009f-c111-470f-a588-91986c1d5388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:07.574879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf009f-c111-470f-a588-91986c1d5388 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:07.575120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf009f-c111-470f-a588-91986c1d5388 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:07.575373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf009f-c111-470f-a588-91986c1d5388 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:07.575987 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf009f-c111-470f-a588-91986c1d5388 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:07.576284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf009f-c111-470f-a588-91986c1d5388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:07.577663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf009f-c111-470f-a588-91986c1d5388 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:07.582506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 493/969] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 9 18:31:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:07.588442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa07756-be23-4fac-a89a-44a491e41938 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:07.690490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa07756-be23-4fac-a89a-44a491e41938 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:07.717876 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:07.725954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba1217-5646-459e-9d54-0275f9fd5591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:07.726691 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba1217-5646-459e-9d54-0275f9fd5591 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:07.727020 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba1217-5646-459e-9d54-0275f9fd5591 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:07.727345 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba1217-5646-459e-9d54-0275f9fd5591 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:07.728034 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba1217-5646-459e-9d54-0275f9fd5591 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:07.730117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba1217-5646-459e-9d54-0275f9fd5591 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:07.730117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba1217-5646-459e-9d54-0275f9fd5591 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:07.739435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 477/970] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 9 18:31:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:07.741223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf009f-c111-470f-a588-91986c1d5388 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:07.784292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcf6f6-4a20-4410-af6a-219154de01da None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:07.784772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcf6f6-4a20-4410-af6a-219154de01da None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:07.785150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcf6f6-4a20-4410-af6a-219154de01da None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:07.787240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 478/971] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:07.792311 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd9214-dbfb-466e-baa1-aa785232ca88 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:07.792591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd9214-dbfb-466e-baa1-aa785232ca88 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:07.792857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd9214-dbfb-466e-baa1-aa785232ca88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:07.811844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-dbfd9214-dbfb-466e-baa1-aa785232ca88 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:07.813009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dbfd9214-dbfb-466e-baa1-aa785232ca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:07.830605 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:07.860249 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf009f-c111-470f-a588-91986c1d5388 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:07.897892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 479/972] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:31:07.898121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644e72c-1605-4bdc-88ad-289243bb237f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:07.898503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644e72c-1605-4bdc-88ad-289243bb237f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:07.898784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644e72c-1605-4bdc-88ad-289243bb237f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:07.899055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644e72c-1605-4bdc-88ad-289243bb237f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:07.899765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644e72c-1605-4bdc-88ad-289243bb237f None admin] RBAC: Adding query filter params (interface=admin, service_id=3288e08e34304e1eb74dbd2ee97f1a49, region_id=RegionOne) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:07.900077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644e72c-1605-4bdc-88ad-289243bb237f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:07.901445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644e72c-1605-4bdc-88ad-289243bb237f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:07.909561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 494/973] 10.222.0.57 () {56 vars in 1249 bytes} [Tue Aug 9 18:31:07 2022] GET /identity/v3/endpoints?service_id=3288e08e34304e1eb74dbd2ee97f1a49&interface=admin®ion_id=RegionOne => generated 193 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:07.912757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd9214-dbfb-466e-baa1-aa785232ca88 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:07.915797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644e72c-1605-4bdc-88ad-289243bb237f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:08.032706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd9214-dbfb-466e-baa1-aa785232ca88 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:08.034054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644e72c-1605-4bdc-88ad-289243bb237f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:08.062649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:08.074132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7a695-1acb-4f17-a0d2-64fe6b6fac05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:08.074854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7a695-1acb-4f17-a0d2-64fe6b6fac05 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:08.075202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7a695-1acb-4f17-a0d2-64fe6b6fac05 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:08.075540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7a695-1acb-4f17-a0d2-64fe6b6fac05 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:08.079774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16a6c4-1a33-4d76-ab92-2f0e7102d941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:08.081322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16a6c4-1a33-4d76-ab92-2f0e7102d941 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:08.081722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16a6c4-1a33-4d76-ab92-2f0e7102d941 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:08.082376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16a6c4-1a33-4d76-ab92-2f0e7102d941 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:08.083335 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16a6c4-1a33-4d76-ab92-2f0e7102d941 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:08.085845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:08.086257 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16a6c4-1a33-4d76-ab92-2f0e7102d941 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:08.090800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7a695-1acb-4f17-a0d2-64fe6b6fac05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:31:08.094103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 480/974] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 9 18:31:07 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-cbf7a695-1acb-4f17-a0d2-64fe6b6fac05 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:31:08.095671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:08.096576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:08.102846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 495/975] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Aug 9 18:31:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:08.103558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16a6c4-1a33-4d76-ab92-2f0e7102d941 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:08.236351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bbb8d-e50e-409a-bb34-3ba6a21692dc None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:08.236944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bbb8d-e50e-409a-bb34-3ba6a21692dc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:08.236944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bbb8d-e50e-409a-bb34-3ba6a21692dc None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:08.238270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 496/976] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:08.244349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e074777-17cc-449b-bd21-e1a513fe763a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:08.245131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e074777-17cc-449b-bd21-e1a513fe763a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:08.245131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e074777-17cc-449b-bd21-e1a513fe763a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:08.265615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2e074777-17cc-449b-bd21-e1a513fe763a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:08.266417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2e074777-17cc-449b-bd21-e1a513fe763a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:08.280708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16a6c4-1a33-4d76-ab92-2f0e7102d941 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:08.320810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c158f-f88b-47e0-84df-5afa4f1deb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:08.321612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c158f-f88b-47e0-84df-5afa4f1deb99 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:08.321903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c158f-f88b-47e0-84df-5afa4f1deb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:08.322463 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c158f-f88b-47e0-84df-5afa4f1deb99 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:08.323141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c158f-f88b-47e0-84df-5afa4f1deb99 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:08.323450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c158f-f88b-47e0-84df-5afa4f1deb99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:08.325255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c158f-f88b-47e0-84df-5afa4f1deb99 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:08.334851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 481/977] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 9 18:31:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:08.340170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c158f-f88b-47e0-84df-5afa4f1deb99 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:08.342988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 497/978] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:31:08.366319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e074777-17cc-449b-bd21-e1a513fe763a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:08.398439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:08.459009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c158f-f88b-47e0-84df-5afa4f1deb99 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:08.484269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e074777-17cc-449b-bd21-e1a513fe763a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:08.503935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f5aba-779c-4909-8d4a-a208700113dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:08.504918 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f5aba-779c-4909-8d4a-a208700113dd None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:08.505343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f5aba-779c-4909-8d4a-a208700113dd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:08.505784 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f5aba-779c-4909-8d4a-a208700113dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:08.516099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:08.522371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f5aba-779c-4909-8d4a-a208700113dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:08.524237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750158d5-9439-418a-9222-8328b7e9ae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:08.525059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f5aba-779c-4909-8d4a-a208700113dd None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:08.525152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750158d5-9439-418a-9222-8328b7e9ae14 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:08.525152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750158d5-9439-418a-9222-8328b7e9ae14 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:08.525457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750158d5-9439-418a-9222-8328b7e9ae14 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:08.526032 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750158d5-9439-418a-9222-8328b7e9ae14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:08.527400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750158d5-9439-418a-9222-8328b7e9ae14 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-5d4f5aba-779c-4909-8d4a-a208700113dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:08.547475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:08.548559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:31:08.549644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:08.549644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:08.549644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:08.549644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 482/979] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:08.555873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f5aba-779c-4909-8d4a-a208700113dd None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:08.621207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 498/980] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:31:08 2022] POST /identity/v3/services => generated 242 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:31:08.688889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:08.693699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f5aba-779c-4909-8d4a-a208700113dd None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:08.744199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794d6c8-bc2c-42ac-9b5b-ffc764ee12dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:08.744931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794d6c8-bc2c-42ac-9b5b-ffc764ee12dd None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:08.745215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794d6c8-bc2c-42ac-9b5b-ffc764ee12dd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:08.745488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794d6c8-bc2c-42ac-9b5b-ffc764ee12dd None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:08.746179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5794d6c8-bc2c-42ac-9b5b-ffc764ee12dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:08.746483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5794d6c8-bc2c-42ac-9b5b-ffc764ee12dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:08.747866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5794d6c8-bc2c-42ac-9b5b-ffc764ee12dd None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:08.765639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 483/981] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:08.766956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750158d5-9439-418a-9222-8328b7e9ae14 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:08.886972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750158d5-9439-418a-9222-8328b7e9ae14 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:08.922629 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2395dad-5b0d-4889-9ce1-e47da50a3420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:08.923269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2395dad-5b0d-4889-9ce1-e47da50a3420 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:08.923551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2395dad-5b0d-4889-9ce1-e47da50a3420 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:08.923826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2395dad-5b0d-4889-9ce1-e47da50a3420 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:08.932311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2395dad-5b0d-4889-9ce1-e47da50a3420 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:08.933830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2395dad-5b0d-4889-9ce1-e47da50a3420 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-d2395dad-5b0d-4889-9ce1-e47da50a3420 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:31:08.945931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:08.946881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:08.947835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:08.947835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:08.947835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:31:08.947835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:31:08.947835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:31:08.947835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:31:08.947835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:31:08.947835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:31:08.947835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 18:31:08.947835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:08.947835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 499/982] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 9 18:31:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:08.953151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794d6c8-bc2c-42ac-9b5b-ffc764ee12dd None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:09.064153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794d6c8-bc2c-42ac-9b5b-ffc764ee12dd None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:09.107072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dea0b8-5272-4789-a285-7de707c5dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:09.107685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dea0b8-5272-4789-a285-7de707c5dc8a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:09.107966 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dea0b8-5272-4789-a285-7de707c5dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:09.108307 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dea0b8-5272-4789-a285-7de707c5dc8a None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:09.109049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dea0b8-5272-4789-a285-7de707c5dc8a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:09.109358 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dea0b8-5272-4789-a285-7de707c5dc8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:09.110829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dea0b8-5272-4789-a285-7de707c5dc8a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:09.129443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 484/983] 10.222.0.57 () {56 vars in 1149 bytes} [Tue Aug 9 18:31:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:09.135721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2395dad-5b0d-4889-9ce1-e47da50a3420 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:09.193627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:09.258792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2395dad-5b0d-4889-9ce1-e47da50a3420 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:09.310468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2aa29-b5ab-43a7-9014-c293cc26ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:09.310468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2aa29-b5ab-43a7-9014-c293cc26ba7c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:09.310468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2aa29-b5ab-43a7-9014-c293cc26ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:09.310468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2aa29-b5ab-43a7-9014-c293cc26ba7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:09.393218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c2aa29-b5ab-43a7-9014-c293cc26ba7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:09.394721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c2aa29-b5ab-43a7-9014-c293cc26ba7c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:09.450618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:09.524572 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 500/984] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:31:09 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:31:09.525851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dea0b8-5272-4789-a285-7de707c5dc8a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:09.545074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:09.631898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dea0b8-5272-4789-a285-7de707c5dc8a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:09.667222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b39f0-64e9-4e6d-93f5-371b017748d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:09.668236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b39f0-64e9-4e6d-93f5-371b017748d6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:09.668705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b39f0-64e9-4e6d-93f5-371b017748d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:09.669158 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b39f0-64e9-4e6d-93f5-371b017748d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:09.685049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b39f0-64e9-4e6d-93f5-371b017748d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:09.686585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b39f0-64e9-4e6d-93f5-371b017748d6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-346b39f0-64e9-4e6d-93f5-371b017748d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:09.701159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:09.702459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:31:09.703438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:09.703438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:09.703438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:09.703438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 485/985] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:09.707953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2aa29-b5ab-43a7-9014-c293cc26ba7c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:09.823734 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2aa29-b5ab-43a7-9014-c293cc26ba7c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:09.860556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81146d-94fd-451e-86cc-321d790f9e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:09.861165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81146d-94fd-451e-86cc-321d790f9e33 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:09.861436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81146d-94fd-451e-86cc-321d790f9e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:09.861827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81146d-94fd-451e-86cc-321d790f9e33 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:09.862477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81146d-94fd-451e-86cc-321d790f9e33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:09.862774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81146d-94fd-451e-86cc-321d790f9e33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:09.864141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81146d-94fd-451e-86cc-321d790f9e33 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:09.882245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 501/986] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:09.883468 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b39f0-64e9-4e6d-93f5-371b017748d6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:09.951652 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:10.008750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b39f0-64e9-4e6d-93f5-371b017748d6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:10.067254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013aca1-2656-44d9-8bf5-add42e0f7b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:10.068417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013aca1-2656-44d9-8bf5-add42e0f7b43 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:10.068899 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013aca1-2656-44d9-8bf5-add42e0f7b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:10.069380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013aca1-2656-44d9-8bf5-add42e0f7b43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:10.081231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013aca1-2656-44d9-8bf5-add42e0f7b43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:10.084540 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013aca1-2656-44d9-8bf5-add42e0f7b43 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-c013aca1-2656-44d9-8bf5-add42e0f7b43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:31:10.105357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:10.106544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:10.107500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:31:10.107500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:31:10.107500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:31:10.107500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:31:10.107500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:31:10.107500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:31:10.107500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:10.107500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:10.108664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 486/987] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:31:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:10.116532 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81146d-94fd-451e-86cc-321d790f9e33 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:10.136036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:10.181536 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819183f0-5d14-445e-a56e-c1c6da801166 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:10.181849 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819183f0-5d14-445e-a56e-c1c6da801166 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:10.182115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819183f0-5d14-445e-a56e-c1c6da801166 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:10.185562 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 487/988] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:31:10.188505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da053d-0005-41c1-83b0-e1e7c8ab673b None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:10.188792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da053d-0005-41c1-83b0-e1e7c8ab673b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:10.189173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da053d-0005-41c1-83b0-e1e7c8ab673b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:10.207615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b3da053d-0005-41c1-83b0-e1e7c8ab673b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:10.209229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b3da053d-0005-41c1-83b0-e1e7c8ab673b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:10.270285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81146d-94fd-451e-86cc-321d790f9e33 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:10.313545 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 488/989] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:31:10.315416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeed8f5-2f97-41ef-bd4c-863000b66f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:10.316245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeed8f5-2f97-41ef-bd4c-863000b66f60 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:10.316637 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeed8f5-2f97-41ef-bd4c-863000b66f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:10.317013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeed8f5-2f97-41ef-bd4c-863000b66f60 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:10.318100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeed8f5-2f97-41ef-bd4c-863000b66f60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:10.318589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeed8f5-2f97-41ef-bd4c-863000b66f60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:10.320937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeed8f5-2f97-41ef-bd4c-863000b66f60 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:10.326334 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da053d-0005-41c1-83b0-e1e7c8ab673b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:10.336255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 502/990] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:31:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:10.343587 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeed8f5-2f97-41ef-bd4c-863000b66f60 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:10.418262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:10.439175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:10.491967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeed8f5-2f97-41ef-bd4c-863000b66f60 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:10.511729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da053d-0005-41c1-83b0-e1e7c8ab673b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:10.555239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf81ed-1766-4a74-86f6-a107433c3b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:10.556305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf81ed-1766-4a74-86f6-a107433c3b45 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:10.556738 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf81ed-1766-4a74-86f6-a107433c3b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:10.559260 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5f355-5a97-44d9-9171-f4bec403bfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:10.559260 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5f355-5a97-44d9-9171-f4bec403bfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:10.559260 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5f355-5a97-44d9-9171-f4bec403bfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:10.559880 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5f355-5a97-44d9-9171-f4bec403bfe4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:10.560121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf81ed-1766-4a74-86f6-a107433c3b45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:10.560561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5f355-5a97-44d9-9171-f4bec403bfe4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:10.560934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf81ed-1766-4a74-86f6-a107433c3b45 None admin] RBAC: Adding query filter params (role.id=039a8882198d4b17911651cf3558be04, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=1255cb6e360244ba9d5cec5cc65a1f8a) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:10.561252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf81ed-1766-4a74-86f6-a107433c3b45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:10.562372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5f355-5a97-44d9-9171-f4bec403bfe4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:10.562755 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf81ed-1766-4a74-86f6-a107433c3b45 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-f9c5f355-5a97-44d9-9171-f4bec403bfe4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:10.573276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:10.574357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:10.575305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 489/991] 10.222.0.57 () {56 vars in 1126 bytes} [Tue Aug 9 18:31:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:10.590063 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 503/992] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:31:10 2022] GET /identity/v3/role_assignments?user.id=1255cb6e360244ba9d5cec5cc65a1f8a&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=039a8882198d4b17911651cf3558be04 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:10.591336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5f355-5a97-44d9-9171-f4bec403bfe4 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:10.723259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5f355-5a97-44d9-9171-f4bec403bfe4 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:10.744862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:10.763835 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d8f5f-0abd-4797-bd81-c3380b392d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:10.765084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d8f5f-0abd-4797-bd81-c3380b392d4c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:10.765084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d8f5f-0abd-4797-bd81-c3380b392d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:10.765084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d8f5f-0abd-4797-bd81-c3380b392d4c None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:10.765716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d8f5f-0abd-4797-bd81-c3380b392d4c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:10.766019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d8f5f-0abd-4797-bd81-c3380b392d4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:10.767418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d8f5f-0abd-4797-bd81-c3380b392d4c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:10.776251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 490/993] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 9 18:31:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:10.783537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf81ed-1766-4a74-86f6-a107433c3b45 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:10.843430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d15bca-a0fa-4161-a0e3-a700c677bb18 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:10.843759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d15bca-a0fa-4161-a0e3-a700c677bb18 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:10.843981 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d15bca-a0fa-4161-a0e3-a700c677bb18 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:10.845441 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 491/994] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:31:10.850688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742498c5-a6b0-4149-bdbb-5566eeaeca7a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:10.850940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742498c5-a6b0-4149-bdbb-5566eeaeca7a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:10.851215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742498c5-a6b0-4149-bdbb-5566eeaeca7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:10.875157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-742498c5-a6b0-4149-bdbb-5566eeaeca7a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:10.876951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-742498c5-a6b0-4149-bdbb-5566eeaeca7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:10.902894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf81ed-1766-4a74-86f6-a107433c3b45 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:10.937881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:10.944212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44172951-64ce-46c4-a1dc-85721c8874b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:10.944670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44172951-64ce-46c4-a1dc-85721c8874b9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:10.951476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44172951-64ce-46c4-a1dc-85721c8874b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:10.951476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44172951-64ce-46c4-a1dc-85721c8874b9 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:10.951476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44172951-64ce-46c4-a1dc-85721c8874b9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:10.951476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44172951-64ce-46c4-a1dc-85721c8874b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:10.951476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44172951-64ce-46c4-a1dc-85721c8874b9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:10.958896 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 504/995] 10.222.0.57 () {56 vars in 1122 bytes} [Tue Aug 9 18:31:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:10.960009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44172951-64ce-46c4-a1dc-85721c8874b9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:10.973482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 492/996] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:31:10.996557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742498c5-a6b0-4149-bdbb-5566eeaeca7a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:11.087600 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44172951-64ce-46c4-a1dc-85721c8874b9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:11.115425 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742498c5-a6b0-4149-bdbb-5566eeaeca7a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:11.128142 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219ed1a-2d6a-47db-b9dd-54427ee69e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:11.128829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219ed1a-2d6a-47db-b9dd-54427ee69e35 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:11.129222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219ed1a-2d6a-47db-b9dd-54427ee69e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:11.129493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219ed1a-2d6a-47db-b9dd-54427ee69e35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:11.130150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:11.130328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219ed1a-2d6a-47db-b9dd-54427ee69e35 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:11.132287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219ed1a-2d6a-47db-b9dd-54427ee69e35 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:11.151919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9831e7-9585-47d6-88b9-f4f1ed78fd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:11.152605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9831e7-9585-47d6-88b9-f4f1ed78fd1d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:11.152880 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9831e7-9585-47d6-88b9-f4f1ed78fd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:11.153148 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9831e7-9585-47d6-88b9-f4f1ed78fd1d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:11.153756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9831e7-9585-47d6-88b9-f4f1ed78fd1d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:11.155100 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9831e7-9585-47d6-88b9-f4f1ed78fd1d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-3e9831e7-9585-47d6-88b9-f4f1ed78fd1d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:11.164475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:11.166713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 18:31:11.167665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:11.170824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 493/997] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:31:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:11.171728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9831e7-9585-47d6-88b9-f4f1ed78fd1d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:11.207626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 505/998] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:31:10 2022] POST /identity/v3/endpoints => generated 325 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:31:11.304563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9831e7-9585-47d6-88b9-f4f1ed78fd1d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:11.358610 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602cec5-e9ed-4a70-887a-e120a2a0e2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:11.359282 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602cec5-e9ed-4a70-887a-e120a2a0e2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:11.359563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602cec5-e9ed-4a70-887a-e120a2a0e2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:11.359832 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602cec5-e9ed-4a70-887a-e120a2a0e2a7 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:11.360470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602cec5-e9ed-4a70-887a-e120a2a0e2a7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:11.360773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602cec5-e9ed-4a70-887a-e120a2a0e2a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:11.363516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602cec5-e9ed-4a70-887a-e120a2a0e2a7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:11.370356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 494/999] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:31:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:11.375947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219ed1a-2d6a-47db-b9dd-54427ee69e35 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:11.441845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:11.495212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219ed1a-2d6a-47db-b9dd-54427ee69e35 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:11.531640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355d2ad-b0d4-4f1b-8ca3-72a0033d9e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:11.532170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355d2ad-b0d4-4f1b-8ca3-72a0033d9e00 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:11.532448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355d2ad-b0d4-4f1b-8ca3-72a0033d9e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:11.532717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355d2ad-b0d4-4f1b-8ca3-72a0033d9e00 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:11.533355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355d2ad-b0d4-4f1b-8ca3-72a0033d9e00 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:11.533684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355d2ad-b0d4-4f1b-8ca3-72a0033d9e00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:11.535349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355d2ad-b0d4-4f1b-8ca3-72a0033d9e00 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:11.545628 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 506/1000] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:31:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:11.546858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602cec5-e9ed-4a70-887a-e120a2a0e2a7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:11.589808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:11.649831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602cec5-e9ed-4a70-887a-e120a2a0e2a7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:11.684494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c967d21-7641-451b-a561-a4474c1ac831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:11.685263 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c967d21-7641-451b-a561-a4474c1ac831 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:11.685263 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c967d21-7641-451b-a561-a4474c1ac831 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:11.685816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c967d21-7641-451b-a561-a4474c1ac831 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:11.686356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c967d21-7641-451b-a561-a4474c1ac831 None admin] RBAC: Adding query filter params (interface=public, service_id=ae42affdd08f435a8b76b02b2392f14f, region_id=RegionOne) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:11.686630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c967d21-7641-451b-a561-a4474c1ac831 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:11.687989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c967d21-7641-451b-a561-a4474c1ac831 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:11.697917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 495/1001] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 9 18:31:11 2022] GET /identity/v3/endpoints?service_id=ae42affdd08f435a8b76b02b2392f14f&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:11.699489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355d2ad-b0d4-4f1b-8ca3-72a0033d9e00 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:11.822888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355d2ad-b0d4-4f1b-8ca3-72a0033d9e00 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:11.846842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:11.861956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cded4-50ab-4a76-81b1-caf3271cacf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:11.862559 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cded4-50ab-4a76-81b1-caf3271cacf0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:11.862838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cded4-50ab-4a76-81b1-caf3271cacf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:11.863108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cded4-50ab-4a76-81b1-caf3271cacf0 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:11.863709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cded4-50ab-4a76-81b1-caf3271cacf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:11.865095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cded4-50ab-4a76-81b1-caf3271cacf0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:11.877615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 507/1002] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 9 18:31:11 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:31:12.674360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5c84f-1ff5-46f4-80df-b1c974804c94 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:12.674891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5c84f-1ff5-46f4-80df-b1c974804c94 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:12.674891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5c84f-1ff5-46f4-80df-b1c974804c94 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:12.676390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 496/1003] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:31:12.682217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033a0bc-69a9-4d4f-923e-cc22fe361dae None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:12.682625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033a0bc-69a9-4d4f-923e-cc22fe361dae None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:12.682848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033a0bc-69a9-4d4f-923e-cc22fe361dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:12.700613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7033a0bc-69a9-4d4f-923e-cc22fe361dae None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:12.701830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7033a0bc-69a9-4d4f-923e-cc22fe361dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:12.769969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 508/1004] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:12 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:12.792132 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5c84f-1ff5-46f4-80df-b1c974804c94 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:12.900374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5c84f-1ff5-46f4-80df-b1c974804c94 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:12.901969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:12.936474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f2a5c-8afd-489e-9ca3-df038d050686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:12.937012 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f2a5c-8afd-489e-9ca3-df038d050686 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:12.937289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f2a5c-8afd-489e-9ca3-df038d050686 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:12.937586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f2a5c-8afd-489e-9ca3-df038d050686 None admin] PATH_INFO: `/v3/services/image` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:12.938328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f2a5c-8afd-489e-9ca3-df038d050686 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:12.939685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f2a5c-8afd-489e-9ca3-df038d050686 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-804f2a5c-8afd-489e-9ca3-df038d050686 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:12.948595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:12.949635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 18:31:12.950515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:12.954209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 497/1005] 10.222.0.57 () {56 vars in 1105 bytes} [Tue Aug 9 18:31:12 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:12.955640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033a0bc-69a9-4d4f-923e-cc22fe361dae None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:13.071971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033a0bc-69a9-4d4f-923e-cc22fe361dae None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:13.087725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:13.108912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576ea81-8b45-419c-b67b-c625bc13a039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:13.109513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ea81-8b45-419c-b67b-c625bc13a039 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:13.109818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ea81-8b45-419c-b67b-c625bc13a039 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:13.110085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ea81-8b45-419c-b67b-c625bc13a039 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:13.110712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576ea81-8b45-419c-b67b-c625bc13a039 None admin] RBAC: Adding query filter params (name=image) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:13.111018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576ea81-8b45-419c-b67b-c625bc13a039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:13.112381 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576ea81-8b45-419c-b67b-c625bc13a039 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:13.122087 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 509/1006] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:13.123405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f2a5c-8afd-489e-9ca3-df038d050686 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:13.232709 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f2a5c-8afd-489e-9ca3-df038d050686 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:13.280991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a43819-8fdb-4e9b-abc3-906db6505f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:13.281627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a43819-8fdb-4e9b-abc3-906db6505f42 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:13.281908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a43819-8fdb-4e9b-abc3-906db6505f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:13.282177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a43819-8fdb-4e9b-abc3-906db6505f42 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:13.282812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a43819-8fdb-4e9b-abc3-906db6505f42 None admin] RBAC: Adding query filter params (type=image) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:13.283113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a43819-8fdb-4e9b-abc3-906db6505f42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:13.284471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a43819-8fdb-4e9b-abc3-906db6505f42 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:13.293566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 498/1007] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:13.297377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4a705-1f5b-4e63-a60b-522c4a0fbb4f None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:13.297737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4a705-1f5b-4e63-a60b-522c4a0fbb4f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:13.298003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4a705-1f5b-4e63-a60b-522c4a0fbb4f None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:13.299377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 510/1008] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:13.305702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c76440-ba7b-4362-b3ea-11974251bb95 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:13.306004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c76440-ba7b-4362-b3ea-11974251bb95 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:13.306254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c76440-ba7b-4362-b3ea-11974251bb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:13.326234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-26c76440-ba7b-4362-b3ea-11974251bb95 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:13.327452 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-26c76440-ba7b-4362-b3ea-11974251bb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:13.366548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:13.423901 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 499/1009] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:31:13.448131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4a705-1f5b-4e63-a60b-522c4a0fbb4f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:13.560778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4a705-1f5b-4e63-a60b-522c4a0fbb4f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:13.606768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b659ee-d064-4cc7-b957-b0ee4835024e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:13.607404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b659ee-d064-4cc7-b957-b0ee4835024e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:13.607685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b659ee-d064-4cc7-b957-b0ee4835024e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:13.607956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b659ee-d064-4cc7-b957-b0ee4835024e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:13.609214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b659ee-d064-4cc7-b957-b0ee4835024e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:13.610690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b659ee-d064-4cc7-b957-b0ee4835024e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:13.616979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 511/1010] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:13.618297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c76440-ba7b-4362-b3ea-11974251bb95 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:13.746976 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c76440-ba7b-4362-b3ea-11974251bb95 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:13.787152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188cd69-74f4-4b81-844a-c8df4174917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:13.787829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188cd69-74f4-4b81-844a-c8df4174917f None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:13.788057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188cd69-74f4-4b81-844a-c8df4174917f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:13.788325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188cd69-74f4-4b81-844a-c8df4174917f None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:13.788983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188cd69-74f4-4b81-844a-c8df4174917f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:13.790454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188cd69-74f4-4b81-844a-c8df4174917f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:13.866254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 500/1011] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:31:13 2022] POST /identity/v3/projects => generated 311 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:31:13.937909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288b0b3-fbfe-4109-9bb7-52666a0d0d37 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:13.938476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288b0b3-fbfe-4109-9bb7-52666a0d0d37 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:13.938476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288b0b3-fbfe-4109-9bb7-52666a0d0d37 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:13.940008 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 512/1012] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:13.946295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e441d9-d904-4ac2-b2b2-186e7320b96a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:13.946610 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e441d9-d904-4ac2-b2b2-186e7320b96a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:13.946854 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e441d9-d904-4ac2-b2b2-186e7320b96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:13.966746 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a0e441d9-d904-4ac2-b2b2-186e7320b96a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:13.968000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a0e441d9-d904-4ac2-b2b2-186e7320b96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:13.977051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:14.048235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 501/1013] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:14.072222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288b0b3-fbfe-4109-9bb7-52666a0d0d37 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:14.129084 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:14.200555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288b0b3-fbfe-4109-9bb7-52666a0d0d37 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:14.237318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a88096-2eae-4ffa-82f6-e31f791ba3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:14.237976 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a88096-2eae-4ffa-82f6-e31f791ba3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:14.238273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a88096-2eae-4ffa-82f6-e31f791ba3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:14.238516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a88096-2eae-4ffa-82f6-e31f791ba3e4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:14.239093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a88096-2eae-4ffa-82f6-e31f791ba3e4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:14.241014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a88096-2eae-4ffa-82f6-e31f791ba3e4 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-46a88096-2eae-4ffa-82f6-e31f791ba3e4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:14.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:14.251180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:14.252052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 513/1014] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:31:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:14.257514 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e441d9-d904-4ac2-b2b2-186e7320b96a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:14.366537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e441d9-d904-4ac2-b2b2-186e7320b96a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:14.403627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b39d21-066e-4036-8818-5be324a89ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:14.403627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b39d21-066e-4036-8818-5be324a89ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:14.403627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b39d21-066e-4036-8818-5be324a89ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:14.403627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b39d21-066e-4036-8818-5be324a89ce8 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:14.404018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b39d21-066e-4036-8818-5be324a89ce8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:14.404018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b39d21-066e-4036-8818-5be324a89ce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:14.405297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b39d21-066e-4036-8818-5be324a89ce8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:14.413871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 502/1015] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:31:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:14.416604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a88096-2eae-4ffa-82f6-e31f791ba3e4 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:14.556281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a88096-2eae-4ffa-82f6-e31f791ba3e4 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:14.566526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:14.600248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98d0bb-fda4-4f79-9439-43257937ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:14.600882 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98d0bb-fda4-4f79-9439-43257937ca52 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:14.601166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98d0bb-fda4-4f79-9439-43257937ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:14.601442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98d0bb-fda4-4f79-9439-43257937ca52 None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:14.602167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98d0bb-fda4-4f79-9439-43257937ca52 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:14.602475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98d0bb-fda4-4f79-9439-43257937ca52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:14.603877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98d0bb-fda4-4f79-9439-43257937ca52 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:14.614438 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 514/1016] 10.222.0.57 () {56 vars in 1112 bytes} [Tue Aug 9 18:31:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:14.615556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b39d21-066e-4036-8818-5be324a89ce8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:14.624696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:14.729977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b39d21-066e-4036-8818-5be324a89ce8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:14.766170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eec9f8-1a62-4782-a8af-71cbf32ab42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:14.766816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eec9f8-1a62-4782-a8af-71cbf32ab42b None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:14.767055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eec9f8-1a62-4782-a8af-71cbf32ab42b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:14.767315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eec9f8-1a62-4782-a8af-71cbf32ab42b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:14.767981 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eec9f8-1a62-4782-a8af-71cbf32ab42b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:14.769345 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eec9f8-1a62-4782-a8af-71cbf32ab42b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:14.861563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 503/1017] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:31:14 2022] POST /identity/v3/endpoints => generated 334 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:31:15.364408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84002d1-a9b2-4e05-bccb-50857e23bc19 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:15.365482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84002d1-a9b2-4e05-bccb-50857e23bc19 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:15.365482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84002d1-a9b2-4e05-bccb-50857e23bc19 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:15.366901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 515/1018] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:15.372159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bf133-f343-4682-8d6d-b6e30d50e06f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:15.372437 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bf133-f343-4682-8d6d-b6e30d50e06f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:15.372714 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bf133-f343-4682-8d6d-b6e30d50e06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:15.394033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b18bf133-f343-4682-8d6d-b6e30d50e06f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:15.395312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b18bf133-f343-4682-8d6d-b6e30d50e06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:15.460046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:15.477255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 504/1019] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:15 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:15.500354 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84002d1-a9b2-4e05-bccb-50857e23bc19 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:15.612479 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84002d1-a9b2-4e05-bccb-50857e23bc19 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:15.648037 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda6d63-afae-4583-a46b-e8c3654dcd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:15.648645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda6d63-afae-4583-a46b-e8c3654dcd5b None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:15.648917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda6d63-afae-4583-a46b-e8c3654dcd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:15.649187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda6d63-afae-4583-a46b-e8c3654dcd5b None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:15.649862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda6d63-afae-4583-a46b-e8c3654dcd5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:15.651228 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda6d63-afae-4583-a46b-e8c3654dcd5b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:15.721945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 516/1020] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:31:15 2022] POST /identity/v3/services => generated 242 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:31:16.026390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855e9ca-46c6-444d-a32d-db1995a6e7a2 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:16.027146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855e9ca-46c6-444d-a32d-db1995a6e7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:16.027146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855e9ca-46c6-444d-a32d-db1995a6e7a2 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:16.028292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 505/1021] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:31:16.033786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5fce7-97c1-4cc5-b94e-ed40b27e8eaa None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:16.034075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5fce7-97c1-4cc5-b94e-ed40b27e8eaa None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:16.034353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5fce7-97c1-4cc5-b94e-ed40b27e8eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:16.054648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-3da5fce7-97c1-4cc5-b94e-ed40b27e8eaa None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:16.055864 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3da5fce7-97c1-4cc5-b94e-ed40b27e8eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:16.088119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:16.157585 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 517/1022] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:16.179073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855e9ca-46c6-444d-a32d-db1995a6e7a2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:16.306032 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855e9ca-46c6-444d-a32d-db1995a6e7a2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:16.340206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00de0e-d3dc-4f41-8c0a-6520c47046ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:16.340821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00de0e-d3dc-4f41-8c0a-6520c47046ab None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:16.341097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00de0e-d3dc-4f41-8c0a-6520c47046ab None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:16.341365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00de0e-d3dc-4f41-8c0a-6520c47046ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:16.342570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00de0e-d3dc-4f41-8c0a-6520c47046ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:16.343943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00de0e-d3dc-4f41-8c0a-6520c47046ab None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:16.345671 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 506/1023] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:16.351837 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5fce7-97c1-4cc5-b94e-ed40b27e8eaa None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:16.452099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5fce7-97c1-4cc5-b94e-ed40b27e8eaa None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:16.489963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b268d-7ef5-40e2-8b42-ea3a21d4afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:16.490567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b268d-7ef5-40e2-8b42-ea3a21d4afdb None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:16.490840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b268d-7ef5-40e2-8b42-ea3a21d4afdb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:16.491103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b268d-7ef5-40e2-8b42-ea3a21d4afdb None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:16.491753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b268d-7ef5-40e2-8b42-ea3a21d4afdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:16.493216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b268d-7ef5-40e2-8b42-ea3a21d4afdb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:16.607060 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 518/1024] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:31:16 2022] POST /identity/v3/users => generated 285 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:31:17.042124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd55358-471a-4cb2-b07c-7ef36678dcfc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:17.042626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd55358-471a-4cb2-b07c-7ef36678dcfc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:17.042626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd55358-471a-4cb2-b07c-7ef36678dcfc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:17.043956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 507/1025] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:17.049146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83b913-5864-48dc-911d-a83bceeb20eb None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:17.049434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83b913-5864-48dc-911d-a83bceeb20eb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:17.049733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83b913-5864-48dc-911d-a83bceeb20eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:17.068828 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-9a83b913-5864-48dc-911d-a83bceeb20eb None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:17.069985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9a83b913-5864-48dc-911d-a83bceeb20eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:17.082101 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:17.148379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 519/1026] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:17.171453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd55358-471a-4cb2-b07c-7ef36678dcfc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:17.230666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:17.281697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd55358-471a-4cb2-b07c-7ef36678dcfc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:17.319972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44197cae-ae75-403d-9384-d90ce030c54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:17.321662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44197cae-ae75-403d-9384-d90ce030c54d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:17.322169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44197cae-ae75-403d-9384-d90ce030c54d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:17.322752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44197cae-ae75-403d-9384-d90ce030c54d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:17.341284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44197cae-ae75-403d-9384-d90ce030c54d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-44197cae-ae75-403d-9384-d90ce030c54d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:31:17.346072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:17.346999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:17.347815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 508/1027] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Aug 9 18:31:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:17.354768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83b913-5864-48dc-911d-a83bceeb20eb None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:17.467367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83b913-5864-48dc-911d-a83bceeb20eb None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:17.502295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c081d-05a6-4b48-b1fc-cbf2115a4c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:17.502907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c081d-05a6-4b48-b1fc-cbf2115a4c96 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:17.503188 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c081d-05a6-4b48-b1fc-cbf2115a4c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:17.503461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c081d-05a6-4b48-b1fc-cbf2115a4c96 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:17.504125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c081d-05a6-4b48-b1fc-cbf2115a4c96 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:17.504494 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c081d-05a6-4b48-b1fc-cbf2115a4c96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:17.506392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c081d-05a6-4b48-b1fc-cbf2115a4c96 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:17.519855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 520/1028] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 9 18:31:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:17.521897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44197cae-ae75-403d-9384-d90ce030c54d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:17.629878 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44197cae-ae75-403d-9384-d90ce030c54d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:17.647130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:17.669035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37148c9b-5fb6-4d1a-ba72-7ecf2911f687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:17.669705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37148c9b-5fb6-4d1a-ba72-7ecf2911f687 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:17.669992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37148c9b-5fb6-4d1a-ba72-7ecf2911f687 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:17.670271 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37148c9b-5fb6-4d1a-ba72-7ecf2911f687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:17.681964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37148c9b-5fb6-4d1a-ba72-7ecf2911f687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:17.683425 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37148c9b-5fb6-4d1a-ba72-7ecf2911f687 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-37148c9b-5fb6-4d1a-ba72-7ecf2911f687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:17.710666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:17.711675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:31:17.712555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:17.712555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:17.712555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:17.713356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 509/1029] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:17.717560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c081d-05a6-4b48-b1fc-cbf2115a4c96 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:17.729663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:17.817203 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba9545-a792-4e90-8f22-a39e68cca3c7 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:17.817490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba9545-a792-4e90-8f22-a39e68cca3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:17.817845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba9545-a792-4e90-8f22-a39e68cca3c7 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:17.819893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 510/1030] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:31:17.825355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ff001-9a5a-43e7-a1d8-5b64eec60be5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:17.825933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ff001-9a5a-43e7-a1d8-5b64eec60be5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:17.826385 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ff001-9a5a-43e7-a1d8-5b64eec60be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:17.833603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c081d-05a6-4b48-b1fc-cbf2115a4c96 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:17.849123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-eb6ff001-9a5a-43e7-a1d8-5b64eec60be5 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:17.850369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-eb6ff001-9a5a-43e7-a1d8-5b64eec60be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:17.873311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb68751-86ea-4111-af66-56380173ff35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:17.873966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb68751-86ea-4111-af66-56380173ff35 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:17.874250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb68751-86ea-4111-af66-56380173ff35 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:17.874516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb68751-86ea-4111-af66-56380173ff35 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:17.875175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb68751-86ea-4111-af66-56380173ff35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:17.875474 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb68751-86ea-4111-af66-56380173ff35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:17.876847 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb68751-86ea-4111-af66-56380173ff35 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:17.892083 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 521/1031] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:17.897414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb68751-86ea-4111-af66-56380173ff35 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:17.938892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 511/1032] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:31:17.961663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ff001-9a5a-43e7-a1d8-5b64eec60be5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:17.983638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:18.039367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb68751-86ea-4111-af66-56380173ff35 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:18.078691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a5b9e-a693-49d5-b8eb-f4d9d02d5925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:18.079483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a5b9e-a693-49d5-b8eb-f4d9d02d5925 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:18.079768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a5b9e-a693-49d5-b8eb-f4d9d02d5925 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:18.080169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a5b9e-a693-49d5-b8eb-f4d9d02d5925 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:18.089096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a5b9e-a693-49d5-b8eb-f4d9d02d5925 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:18.097197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a5b9e-a693-49d5-b8eb-f4d9d02d5925 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:18.098721 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-911a5b9e-a693-49d5-b8eb-f4d9d02d5925 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:31:18.103321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:18.104300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:18.105240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:18.105240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:18.105240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:31:18.105240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:31:18.105240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:31:18.105240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:31:18.105240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:31:18.105240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:31:18.105240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:31:18.105240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:18.106363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ff001-9a5a-43e7-a1d8-5b64eec60be5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:18.109571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 522/1033] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 9 18:31:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:18.110451 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a5b9e-a693-49d5-b8eb-f4d9d02d5925 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:18.167605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddef737-e292-4647-a71a-64f13941772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:18.168587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddef737-e292-4647-a71a-64f13941772c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:18.169087 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddef737-e292-4647-a71a-64f13941772c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:18.169617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddef737-e292-4647-a71a-64f13941772c None admin] PATH_INFO: `/v3/services/image` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:18.170691 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddef737-e292-4647-a71a-64f13941772c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:18.173285 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddef737-e292-4647-a71a-64f13941772c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-bddef737-e292-4647-a71a-64f13941772c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:18.191569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:18.192862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:18.194113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 512/1034] 10.222.0.57 () {56 vars in 1105 bytes} [Tue Aug 9 18:31:17 2022] GET /identity/v3/services/image => generated 86 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:18.202218 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddef737-e292-4647-a71a-64f13941772c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:18.231934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a5b9e-a693-49d5-b8eb-f4d9d02d5925 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:18.259463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:18.272236 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af684d-c275-48b3-8a09-411b4c3e1520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:18.272771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af684d-c275-48b3-8a09-411b4c3e1520 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:18.273046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af684d-c275-48b3-8a09-411b4c3e1520 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:18.274735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af684d-c275-48b3-8a09-411b4c3e1520 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:18.274735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af684d-c275-48b3-8a09-411b4c3e1520 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:18.274735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af684d-c275-48b3-8a09-411b4c3e1520 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:18.276356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af684d-c275-48b3-8a09-411b4c3e1520 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:18.303300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 523/1035] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 9 18:31:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:18.304393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af684d-c275-48b3-8a09-411b4c3e1520 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:18.331479 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddef737-e292-4647-a71a-64f13941772c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:18.369753 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f42062-2d2b-4a41-8ad7-2f618761411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:18.370389 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f42062-2d2b-4a41-8ad7-2f618761411f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:18.370666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f42062-2d2b-4a41-8ad7-2f618761411f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:18.370934 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f42062-2d2b-4a41-8ad7-2f618761411f None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:18.371671 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f42062-2d2b-4a41-8ad7-2f618761411f None admin] RBAC: Adding query filter params (name=image) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:18.372082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f42062-2d2b-4a41-8ad7-2f618761411f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:18.373554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f42062-2d2b-4a41-8ad7-2f618761411f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:18.379251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 513/1036] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:18.385092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f42062-2d2b-4a41-8ad7-2f618761411f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:18.418119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af684d-c275-48b3-8a09-411b4c3e1520 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:18.427253 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:18.455429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13165ec-b7c7-49d1-b84c-85e8f3e6089f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:18.456033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13165ec-b7c7-49d1-b84c-85e8f3e6089f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:18.456452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13165ec-b7c7-49d1-b84c-85e8f3e6089f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:18.456590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13165ec-b7c7-49d1-b84c-85e8f3e6089f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:18.527097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f42062-2d2b-4a41-8ad7-2f618761411f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:18.528422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:18.550760 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13165ec-b7c7-49d1-b84c-85e8f3e6089f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:18.553204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13165ec-b7c7-49d1-b84c-85e8f3e6089f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:18.586895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa383b6-b692-4c20-b7b6-88e549a292df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:18.587520 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa383b6-b692-4c20-b7b6-88e549a292df None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:18.587801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa383b6-b692-4c20-b7b6-88e549a292df None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:18.588072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa383b6-b692-4c20-b7b6-88e549a292df None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:18.588716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa383b6-b692-4c20-b7b6-88e549a292df None admin] RBAC: Adding query filter params (type=image) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:18.589022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa383b6-b692-4c20-b7b6-88e549a292df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:18.590427 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa383b6-b692-4c20-b7b6-88e549a292df None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:18.596065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 514/1037] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:18.602607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa383b6-b692-4c20-b7b6-88e549a292df None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:18.684427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 524/1038] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:31:18 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:31:18.691215 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13165ec-b7c7-49d1-b84c-85e8f3e6089f None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:18.701300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:18.710992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa383b6-b692-4c20-b7b6-88e549a292df None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:18.749942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d64b9-9b1a-4d13-86c1-f4bcb406be35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:18.750549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d64b9-9b1a-4d13-86c1-f4bcb406be35 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:18.750825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d64b9-9b1a-4d13-86c1-f4bcb406be35 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:18.751089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d64b9-9b1a-4d13-86c1-f4bcb406be35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:18.751794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d64b9-9b1a-4d13-86c1-f4bcb406be35 None admin] RBAC: Adding query filter params (interface=public, service_id=b14b68944c924058aac3daa442f75886, region_id=RegionOne) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:18.752259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d64b9-9b1a-4d13-86c1-f4bcb406be35 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:18.753668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d64b9-9b1a-4d13-86c1-f4bcb406be35 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:18.759618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 515/1039] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 9 18:31:18 2022] GET /identity/v3/endpoints?service_id=b14b68944c924058aac3daa442f75886&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:18.765070 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d64b9-9b1a-4d13-86c1-f4bcb406be35 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:18.793789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:18.809365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13165ec-b7c7-49d1-b84c-85e8f3e6089f None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:18.853126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8429f-6546-4de5-912a-7c3af4a13836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:18.853786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8429f-6546-4de5-912a-7c3af4a13836 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:18.854065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8429f-6546-4de5-912a-7c3af4a13836 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:18.854339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8429f-6546-4de5-912a-7c3af4a13836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:18.864893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8429f-6546-4de5-912a-7c3af4a13836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:18.866385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8429f-6546-4de5-912a-7c3af4a13836 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:18.872739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d64b9-9b1a-4d13-86c1-f4bcb406be35 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-09c8429f-6546-4de5-912a-7c3af4a13836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:18.881533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:18.882526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:31:18.883440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:18.883440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:18.883440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:18.883440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 525/1040] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:18.883950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa084f-3826-415f-8eac-0c7c484fa827 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:18.884119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa084f-3826-415f-8eac-0c7c484fa827 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:18.884375 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa084f-3826-415f-8eac-0c7c484fa827 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:18.885628 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 526/1041] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:18.888090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa084f-3826-415f-8eac-0c7c484fa827 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:18.912958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4f25c-f334-460c-8d27-b43064a38c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:18.913591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4f25c-f334-460c-8d27-b43064a38c7f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:18.913872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4f25c-f334-460c-8d27-b43064a38c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:18.914139 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4f25c-f334-460c-8d27-b43064a38c7f None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:18.914741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4f25c-f334-460c-8d27-b43064a38c7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:18.916092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4f25c-f334-460c-8d27-b43064a38c7f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:18.923185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 516/1042] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 9 18:31:18 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:31:18.924378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b529d-f46b-428a-8816-0feaca4ed48b None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:18.924654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b529d-f46b-428a-8816-0feaca4ed48b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:18.924923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b529d-f46b-428a-8816-0feaca4ed48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:18.948678 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d04b529d-f46b-428a-8816-0feaca4ed48b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:18.949907 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d04b529d-f46b-428a-8816-0feaca4ed48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:18.967312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:19.009090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa084f-3826-415f-8eac-0c7c484fa827 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:19.034469 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 517/1043] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:19.058693 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e6d28-a62a-4c95-b6af-14d3f2398458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:19.061657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e6d28-a62a-4c95-b6af-14d3f2398458 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:19.061657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e6d28-a62a-4c95-b6af-14d3f2398458 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:19.061657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e6d28-a62a-4c95-b6af-14d3f2398458 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:19.061657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e6d28-a62a-4c95-b6af-14d3f2398458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:19.061657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e6d28-a62a-4c95-b6af-14d3f2398458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:19.062043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b529d-f46b-428a-8816-0feaca4ed48b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:19.062281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e6d28-a62a-4c95-b6af-14d3f2398458 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:19.070955 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:19.078231 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 527/1044] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:19.084036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e6d28-a62a-4c95-b6af-14d3f2398458 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:19.181437 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b529d-f46b-428a-8816-0feaca4ed48b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:19.207229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e6d28-a62a-4c95-b6af-14d3f2398458 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:19.219589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48aeab-d0f2-4bf1-b21a-552909c8d8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:19.220227 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48aeab-d0f2-4bf1-b21a-552909c8d8fd None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:19.220506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48aeab-d0f2-4bf1-b21a-552909c8d8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:19.220775 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48aeab-d0f2-4bf1-b21a-552909c8d8fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:19.232679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48aeab-d0f2-4bf1-b21a-552909c8d8fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-0f48aeab-d0f2-4bf1-b21a-552909c8d8fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:31:19.237591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:19.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:19.240019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 518/1045] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:31:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:19.245328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48aeab-d0f2-4bf1-b21a-552909c8d8fd None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:19.248713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0626504-ea4f-4b54-87d3-ad44bc144b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:19.249357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0626504-ea4f-4b54-87d3-ad44bc144b08 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:19.249672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0626504-ea4f-4b54-87d3-ad44bc144b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:19.249944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0626504-ea4f-4b54-87d3-ad44bc144b08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:19.258185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0626504-ea4f-4b54-87d3-ad44bc144b08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:19.259099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:19.259938 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0626504-ea4f-4b54-87d3-ad44bc144b08 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f0626504-ea4f-4b54-87d3-ad44bc144b08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:31:19.277285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:19.278394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:19.279290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:31:19.279290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:31:19.279290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:31:19.279290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:31:19.279290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:31:19.279290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:31:19.279290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:19.279290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:19.279290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 528/1046] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:31:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:19.284468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0626504-ea4f-4b54-87d3-ad44bc144b08 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:19.355247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:19.377797 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48aeab-d0f2-4bf1-b21a-552909c8d8fd None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:19.404897 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0626504-ea4f-4b54-87d3-ad44bc144b08 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:19.419466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9a924-48a2-415f-9bb6-41367fe170a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:19.420122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9a924-48a2-415f-9bb6-41367fe170a5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:19.420448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9a924-48a2-415f-9bb6-41367fe170a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:19.420721 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9a924-48a2-415f-9bb6-41367fe170a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:19.421391 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9a924-48a2-415f-9bb6-41367fe170a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:19.421822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9a924-48a2-415f-9bb6-41367fe170a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:19.423196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9a924-48a2-415f-9bb6-41367fe170a5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:19.433267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 519/1047] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:31:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:19.439476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9a924-48a2-415f-9bb6-41367fe170a5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:19.451743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94376f53-f54d-42d1-a5cd-094d06b7ce43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:19.452358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94376f53-f54d-42d1-a5cd-094d06b7ce43 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:19.452933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94376f53-f54d-42d1-a5cd-094d06b7ce43 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:19.453224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94376f53-f54d-42d1-a5cd-094d06b7ce43 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:19.454233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94376f53-f54d-42d1-a5cd-094d06b7ce43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:19.454538 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94376f53-f54d-42d1-a5cd-094d06b7ce43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:19.456544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94376f53-f54d-42d1-a5cd-094d06b7ce43 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:19.478209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 529/1048] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:31:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:19.479171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94376f53-f54d-42d1-a5cd-094d06b7ce43 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:19.524835 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:19.561941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9a924-48a2-415f-9bb6-41367fe170a5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:19.592345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94376f53-f54d-42d1-a5cd-094d06b7ce43 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:19.602285 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b155247-c8d7-497b-b933-cc34b17e0c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:19.603038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b155247-c8d7-497b-b933-cc34b17e0c64 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:19.603313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b155247-c8d7-497b-b933-cc34b17e0c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:19.603737 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b155247-c8d7-497b-b933-cc34b17e0c64 None admin] PATH_INFO: `/v3/users/54b21288dd78453a89ca6ffea624db83` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:19.612082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:19.619780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b155247-c8d7-497b-b933-cc34b17e0c64 None admin] RBAC: Authorizing `identity:get_user(user_id=54b21288dd78453a89ca6ffea624db83)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:19.621226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b155247-c8d7-497b-b933-cc34b17e0c64 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:19.623092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 520/1049] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:31:19 2022] GET /identity/v3/users/54b21288dd78453a89ca6ffea624db83 => generated 285 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:19.629163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b155247-c8d7-497b-b933-cc34b17e0c64 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:19.631233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05814fe6-7872-4dc3-810e-b3cf2bead27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:19.631842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05814fe6-7872-4dc3-810e-b3cf2bead27d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:19.632124 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05814fe6-7872-4dc3-810e-b3cf2bead27d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:19.632396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05814fe6-7872-4dc3-810e-b3cf2bead27d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:19.633192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05814fe6-7872-4dc3-810e-b3cf2bead27d None admin] RBAC: Adding query filter params (role.id=039a8882198d4b17911651cf3558be04, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=e2ecd7d1cce644b0982cc23740537933) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:19.633502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05814fe6-7872-4dc3-810e-b3cf2bead27d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:19.634999 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05814fe6-7872-4dc3-810e-b3cf2bead27d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:19.641566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 530/1050] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:31:19 2022] GET /identity/v3/role_assignments?user.id=e2ecd7d1cce644b0982cc23740537933&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=039a8882198d4b17911651cf3558be04 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:19.750921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b155247-c8d7-497b-b933-cc34b17e0c64 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:19.784487 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:19.793413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca793da5-1098-4eea-9c9b-4ec92ea8e3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:19.794105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca793da5-1098-4eea-9c9b-4ec92ea8e3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:19.794383 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca793da5-1098-4eea-9c9b-4ec92ea8e3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:19.794650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca793da5-1098-4eea-9c9b-4ec92ea8e3d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:19.870478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca793da5-1098-4eea-9c9b-4ec92ea8e3d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:19.871927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca793da5-1098-4eea-9c9b-4ec92ea8e3d8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:19.979373 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 521/1051] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:31:19 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 18:31:19.981026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05814fe6-7872-4dc3-810e-b3cf2bead27d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:20.095290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05814fe6-7872-4dc3-810e-b3cf2bead27d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:20.135733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba1bfc-52fa-44c5-8bd5-79c24c3355d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:20.136388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba1bfc-52fa-44c5-8bd5-79c24c3355d0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:20.136669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba1bfc-52fa-44c5-8bd5-79c24c3355d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:20.137000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba1bfc-52fa-44c5-8bd5-79c24c3355d0 None admin] PATH_INFO: `/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:20.138273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba1bfc-52fa-44c5-8bd5-79c24c3355d0 None admin] RBAC: Authorizing `identity:get_project(project_id=6d5860cfa51e40f5b1c6fe9425798e9f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:20.139809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba1bfc-52fa-44c5-8bd5-79c24c3355d0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:20.146969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 531/1052] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:31:19 2022] GET /identity/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:20.148294 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca793da5-1098-4eea-9c9b-4ec92ea8e3d8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:20.179126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:20.254156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca793da5-1098-4eea-9c9b-4ec92ea8e3d8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:20.296405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c557088-0b35-4cba-ae2e-574a76e9d748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:20.297015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c557088-0b35-4cba-ae2e-574a76e9d748 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:20.297295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c557088-0b35-4cba-ae2e-574a76e9d748 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:20.297594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c557088-0b35-4cba-ae2e-574a76e9d748 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:20.298332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c557088-0b35-4cba-ae2e-574a76e9d748 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=6d5860cfa51e40f5b1c6fe9425798e9f, user.id=54b21288dd78453a89ca6ffea624db83) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:20.298703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c557088-0b35-4cba-ae2e-574a76e9d748 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:20.300173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c557088-0b35-4cba-ae2e-574a76e9d748 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:20.306962 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 522/1053] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:31:20 2022] GET /identity/v3/role_assignments?user.id=54b21288dd78453a89ca6ffea624db83&scope.project.id=6d5860cfa51e40f5b1c6fe9425798e9f&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:21.029175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce3343-a548-4fb5-8fb2-5b5fb246de31 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:21.029756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce3343-a548-4fb5-8fb2-5b5fb246de31 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:21.029756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce3343-a548-4fb5-8fb2-5b5fb246de31 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:21.031135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 532/1054] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:21.036200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cfc34-316e-4297-8f47-07c0cc50fa6f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:21.037086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cfc34-316e-4297-8f47-07c0cc50fa6f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:21.037492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cfc34-316e-4297-8f47-07c0cc50fa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:21.056855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-db6cfc34-316e-4297-8f47-07c0cc50fa6f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:21.062612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-db6cfc34-316e-4297-8f47-07c0cc50fa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:21.169576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 523/1055] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:21.192389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce3343-a548-4fb5-8fb2-5b5fb246de31 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:21.247259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:21.299533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce3343-a548-4fb5-8fb2-5b5fb246de31 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:21.336649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993ea2f-3111-4d7d-bc7c-a48d2c111973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:21.337284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993ea2f-3111-4d7d-bc7c-a48d2c111973 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:21.337588 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993ea2f-3111-4d7d-bc7c-a48d2c111973 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:21.337864 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993ea2f-3111-4d7d-bc7c-a48d2c111973 None admin] PATH_INFO: `/v3/services/image` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:21.338444 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993ea2f-3111-4d7d-bc7c-a48d2c111973 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:21.339879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993ea2f-3111-4d7d-bc7c-a48d2c111973 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b993ea2f-3111-4d7d-bc7c-a48d2c111973 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:21.348727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:21.350486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 18:31:21.351316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:21.353892 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 533/1056] 10.222.0.57 () {56 vars in 1105 bytes} [Tue Aug 9 18:31:21 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:21.356419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cfc34-316e-4297-8f47-07c0cc50fa6f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:21.380513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:21.461358 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cfc34-316e-4297-8f47-07c0cc50fa6f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:21.496741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da92337-1fc7-4527-838e-21904be9daf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:21.497351 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da92337-1fc7-4527-838e-21904be9daf9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:21.497660 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da92337-1fc7-4527-838e-21904be9daf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:21.497935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da92337-1fc7-4527-838e-21904be9daf9 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:21.498568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da92337-1fc7-4527-838e-21904be9daf9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:21.498872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da92337-1fc7-4527-838e-21904be9daf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:21.500226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da92337-1fc7-4527-838e-21904be9daf9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:21.505742 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 524/1057] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:21.511269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993ea2f-3111-4d7d-bc7c-a48d2c111973 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:21.619898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993ea2f-3111-4d7d-bc7c-a48d2c111973 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:21.633734 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:21.654601 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd3257-c097-4ee9-88d1-b3f92e3bf2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:21.655213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd3257-c097-4ee9-88d1-b3f92e3bf2fe None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:21.655464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd3257-c097-4ee9-88d1-b3f92e3bf2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:21.655736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd3257-c097-4ee9-88d1-b3f92e3bf2fe None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:21.656365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd3257-c097-4ee9-88d1-b3f92e3bf2fe None admin] RBAC: Adding query filter params (type=image) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:21.656667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd3257-c097-4ee9-88d1-b3f92e3bf2fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:21.658098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd3257-c097-4ee9-88d1-b3f92e3bf2fe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:21.663588 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 534/1058] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:21.668067 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99565c8a-b102-4ec8-b330-2e3f0bbe6d83 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:21.668067 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99565c8a-b102-4ec8-b330-2e3f0bbe6d83 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:21.668271 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99565c8a-b102-4ec8-b330-2e3f0bbe6d83 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:21.669099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd3257-c097-4ee9-88d1-b3f92e3bf2fe None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:21.669806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 525/1059] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:21.674557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624c86c-3570-418c-b4d6-d95899acc228 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:21.674837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624c86c-3570-418c-b4d6-d95899acc228 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:21.675111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624c86c-3570-418c-b4d6-d95899acc228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:21.693949 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b624c86c-3570-418c-b4d6-d95899acc228 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:21.695324 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b624c86c-3570-418c-b4d6-d95899acc228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:21.767646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 526/1060] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:31:21.779974 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd3257-c097-4ee9-88d1-b3f92e3bf2fe None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:21.790565 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624c86c-3570-418c-b4d6-d95899acc228 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:21.814044 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:21.818385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046a4ea-43af-4e60-9319-ae1988ce8648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:21.818994 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046a4ea-43af-4e60-9319-ae1988ce8648 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:21.819274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046a4ea-43af-4e60-9319-ae1988ce8648 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:21.819538 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046a4ea-43af-4e60-9319-ae1988ce8648 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:21.820187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046a4ea-43af-4e60-9319-ae1988ce8648 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:21.821649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046a4ea-43af-4e60-9319-ae1988ce8648 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:21.901246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624c86c-3570-418c-b4d6-d95899acc228 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:21.933140 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 535/1061] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:31:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:31:21.943384 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece8fe9-f9d4-43c1-8bbe-0760a634abd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:21.944038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece8fe9-f9d4-43c1-8bbe-0760a634abd1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:21.944319 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece8fe9-f9d4-43c1-8bbe-0760a634abd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:21.944591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece8fe9-f9d4-43c1-8bbe-0760a634abd1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:21.956961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece8fe9-f9d4-43c1-8bbe-0760a634abd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-fece8fe9-f9d4-43c1-8bbe-0760a634abd1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:31:21.962601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:21.963614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:21.970004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 527/1062] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Aug 9 18:31:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:21.970819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046a4ea-43af-4e60-9319-ae1988ce8648 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:22.009672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:22.084811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046a4ea-43af-4e60-9319-ae1988ce8648 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:22.126241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cc2c3-4709-4358-afac-9467a05d6cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:22.127118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cc2c3-4709-4358-afac-9467a05d6cdd None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:22.127577 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cc2c3-4709-4358-afac-9467a05d6cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:22.127852 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cc2c3-4709-4358-afac-9467a05d6cdd None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:22.128530 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50cc2c3-4709-4358-afac-9467a05d6cdd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:22.128832 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50cc2c3-4709-4358-afac-9467a05d6cdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:22.131134 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50cc2c3-4709-4358-afac-9467a05d6cdd None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:22.142365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 536/1063] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 9 18:31:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:22.147849 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece8fe9-f9d4-43c1-8bbe-0760a634abd1 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:22.221645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:22.253639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece8fe9-f9d4-43c1-8bbe-0760a634abd1 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:22.295355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733436d-351d-42c0-ac6c-ddd60e403457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:22.296036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733436d-351d-42c0-ac6c-ddd60e403457 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:22.296314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733436d-351d-42c0-ac6c-ddd60e403457 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:22.296580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733436d-351d-42c0-ac6c-ddd60e403457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:22.308743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733436d-351d-42c0-ac6c-ddd60e403457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:22.310226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733436d-351d-42c0-ac6c-ddd60e403457 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-c733436d-351d-42c0-ac6c-ddd60e403457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:22.325128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:22.326149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:31:22.327036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:22.327036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:22.327036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:22.327679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 528/1064] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:22.332485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cc2c3-4709-4358-afac-9467a05d6cdd None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:22.376704 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a447366-49de-4f78-8bac-2de1db312a74 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:22.377233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a447366-49de-4f78-8bac-2de1db312a74 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:22.377349 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a447366-49de-4f78-8bac-2de1db312a74 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:22.378845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 529/1065] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:31:22.384363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06aeae-55c5-45e7-b33c-78ee4b14af5d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:22.384650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06aeae-55c5-45e7-b33c-78ee4b14af5d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:22.384925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06aeae-55c5-45e7-b33c-78ee4b14af5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:22.403464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ef06aeae-55c5-45e7-b33c-78ee4b14af5d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:22.404662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ef06aeae-55c5-45e7-b33c-78ee4b14af5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:22.442625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:22.447885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cc2c3-4709-4358-afac-9467a05d6cdd None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:22.484366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 530/1066] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:31:22.484543 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff20eb6-074a-484f-ac64-2816f93c28d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:22.485009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff20eb6-074a-484f-ac64-2816f93c28d1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:22.485284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff20eb6-074a-484f-ac64-2816f93c28d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:22.485717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff20eb6-074a-484f-ac64-2816f93c28d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:22.486299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff20eb6-074a-484f-ac64-2816f93c28d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:22.486596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff20eb6-074a-484f-ac64-2816f93c28d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:22.487953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff20eb6-074a-484f-ac64-2816f93c28d1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:22.504949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 537/1067] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:22.506708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06aeae-55c5-45e7-b33c-78ee4b14af5d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:22.513327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff20eb6-074a-484f-ac64-2816f93c28d1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:22.576619 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:22.626937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06aeae-55c5-45e7-b33c-78ee4b14af5d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:22.643592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff20eb6-074a-484f-ac64-2816f93c28d1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:22.662414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d85276-9215-411e-a7d2-cada33d52d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:22.663027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d85276-9215-411e-a7d2-cada33d52d12 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:22.663301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d85276-9215-411e-a7d2-cada33d52d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:22.663568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d85276-9215-411e-a7d2-cada33d52d12 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:22.671932 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d85276-9215-411e-a7d2-cada33d52d12 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:22.673375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d85276-9215-411e-a7d2-cada33d52d12 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:22.683410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2d537-ca7a-4c1d-baa7-5fbd0c119178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:22.684089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2d537-ca7a-4c1d-baa7-5fbd0c119178 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:22.684370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2d537-ca7a-4c1d-baa7-5fbd0c119178 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:22.684640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2d537-ca7a-4c1d-baa7-5fbd0c119178 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-05d85276-9215-411e-a7d2-cada33d52d12 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:31:22.685643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:22.686560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:22.688251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:22.688251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:22.688251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:31:22.688251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:31:22.688251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:31:22.688251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:31:22.688251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:31:22.688251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:31:22.688251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:31:22.688251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:22.688251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 531/1068] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 9 18:31:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:22.693207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d85276-9215-411e-a7d2-cada33d52d12 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:22.697944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2d537-ca7a-4c1d-baa7-5fbd0c119178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-42e2d537-ca7a-4c1d-baa7-5fbd0c119178 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:31:22.703726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:22.705599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:22.706677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 538/1069] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:31:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:22.711243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2d537-ca7a-4c1d-baa7-5fbd0c119178 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:22.730709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:22.806629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d85276-9215-411e-a7d2-cada33d52d12 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:22.818639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2d537-ca7a-4c1d-baa7-5fbd0c119178 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:22.822212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:22.842970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee52f6-252d-4dfb-a399-8cc2b15749d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:22.843566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee52f6-252d-4dfb-a399-8cc2b15749d2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:22.843841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee52f6-252d-4dfb-a399-8cc2b15749d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:22.844109 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee52f6-252d-4dfb-a399-8cc2b15749d2 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:22.844880 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee52f6-252d-4dfb-a399-8cc2b15749d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:22.845187 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee52f6-252d-4dfb-a399-8cc2b15749d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:22.846684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee52f6-252d-4dfb-a399-8cc2b15749d2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:22.853831 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac21835-01c0-4352-980b-d0adc564430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:22.854434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac21835-01c0-4352-980b-d0adc564430e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:22.854710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac21835-01c0-4352-980b-d0adc564430e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:22.854973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac21835-01c0-4352-980b-d0adc564430e None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:22.855651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac21835-01c0-4352-980b-d0adc564430e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:22.855944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac21835-01c0-4352-980b-d0adc564430e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:22.857286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac21835-01c0-4352-980b-d0adc564430e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:22.866159 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 539/1070] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:31:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:22.867367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 532/1071] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 9 18:31:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:22.872191 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac21835-01c0-4352-980b-d0adc564430e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:22.872625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee52f6-252d-4dfb-a399-8cc2b15749d2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:22.953972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:22.975120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac21835-01c0-4352-980b-d0adc564430e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:22.979803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee52f6-252d-4dfb-a399-8cc2b15749d2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:23.014925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d130bad-d06f-4cc3-be47-0d011cd69511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:23.015547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d130bad-d06f-4cc3-be47-0d011cd69511 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:23.015912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d130bad-d06f-4cc3-be47-0d011cd69511 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:23.016184 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d130bad-d06f-4cc3-be47-0d011cd69511 None admin] PATH_INFO: `/v3/users/54b21288dd78453a89ca6ffea624db83` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:23.016279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee1c43-a928-4bcf-b479-d960394c96c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:23.016881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee1c43-a928-4bcf-b479-d960394c96c7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:23.017161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee1c43-a928-4bcf-b479-d960394c96c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:23.017317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d130bad-d06f-4cc3-be47-0d011cd69511 None admin] RBAC: Authorizing `identity:get_user(user_id=54b21288dd78453a89ca6ffea624db83)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:23.017465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee1c43-a928-4bcf-b479-d960394c96c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:23.018956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d130bad-d06f-4cc3-be47-0d011cd69511 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:23.025627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 540/1072] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:31:22 2022] GET /identity/v3/users/54b21288dd78453a89ca6ffea624db83 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:23.026322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d130bad-d06f-4cc3-be47-0d011cd69511 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:23.028065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee1c43-a928-4bcf-b479-d960394c96c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:23.029512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee1c43-a928-4bcf-b479-d960394c96c7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-eaee1c43-a928-4bcf-b479-d960394c96c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:23.044974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:23.046081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:31:23.047268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:23.047268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:23.047268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:23.047268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 533/1073] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:23.051648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee1c43-a928-4bcf-b479-d960394c96c7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:23.080525 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:23.151612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d130bad-d06f-4cc3-be47-0d011cd69511 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:23.175366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee1c43-a928-4bcf-b479-d960394c96c7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:23.190752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9436b3-adcb-4a10-b868-e2e15a9803e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:23.191376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9436b3-adcb-4a10-b868-e2e15a9803e0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:23.191659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9436b3-adcb-4a10-b868-e2e15a9803e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:23.191929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9436b3-adcb-4a10-b868-e2e15a9803e0 None admin] PATH_INFO: `/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:23.193050 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9436b3-adcb-4a10-b868-e2e15a9803e0 None admin] RBAC: Authorizing `identity:get_project(project_id=6d5860cfa51e40f5b1c6fe9425798e9f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:23.194513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9436b3-adcb-4a10-b868-e2e15a9803e0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:23.196324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 541/1074] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:31:23 2022] GET /identity/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:23.202045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9436b3-adcb-4a10-b868-e2e15a9803e0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:23.212780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805fe98-9cba-4b66-9c0e-7b3f358e3d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:23.213689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805fe98-9cba-4b66-9c0e-7b3f358e3d35 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:23.214144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805fe98-9cba-4b66-9c0e-7b3f358e3d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:23.214587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805fe98-9cba-4b66-9c0e-7b3f358e3d35 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:23.215414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805fe98-9cba-4b66-9c0e-7b3f358e3d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:23.215888 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805fe98-9cba-4b66-9c0e-7b3f358e3d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:23.217427 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805fe98-9cba-4b66-9c0e-7b3f358e3d35 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:23.219624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:23.230023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 534/1075] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:23.235624 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805fe98-9cba-4b66-9c0e-7b3f358e3d35 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:23.312126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9436b3-adcb-4a10-b868-e2e15a9803e0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:23.333772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:23.348313 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4303902-08b5-4c52-87d4-1388c5bce90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:23.349509 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4303902-08b5-4c52-87d4-1388c5bce90c None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:23.349817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4303902-08b5-4c52-87d4-1388c5bce90c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:23.350089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4303902-08b5-4c52-87d4-1388c5bce90c None admin] PATH_INFO: `/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f/users/54b21288dd78453a89ca6ffea624db83/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:23.350544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805fe98-9cba-4b66-9c0e-7b3f358e3d35 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:23.352034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4303902-08b5-4c52-87d4-1388c5bce90c None admin] RBAC: Authorizing `identity:create_grant(project_id=6d5860cfa51e40f5b1c6fe9425798e9f, user_id=54b21288dd78453a89ca6ffea624db83, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:23.353487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4303902-08b5-4c52-87d4-1388c5bce90c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:23.384696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762f2fa-e95d-402e-bf4d-d8f3823aab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:23.385325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762f2fa-e95d-402e-bf4d-d8f3823aab9b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:23.385689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762f2fa-e95d-402e-bf4d-d8f3823aab9b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:23.385965 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762f2fa-e95d-402e-bf4d-d8f3823aab9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:23.392532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762f2fa-e95d-402e-bf4d-d8f3823aab9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:23.394013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762f2fa-e95d-402e-bf4d-d8f3823aab9b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-5762f2fa-e95d-402e-bf4d-d8f3823aab9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:31:23.405370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:23.406430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:23.407863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:31:23.407863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:31:23.407863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:31:23.407863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:31:23.407863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:31:23.407863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:31:23.407863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:23.407863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:23.407863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 535/1076] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:31:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:23.412802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762f2fa-e95d-402e-bf4d-d8f3823aab9b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:23.461718 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 542/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:31:23 2022] PUT /identity/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f/users/54b21288dd78453a89ca6ffea624db83/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:31:23.527314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762f2fa-e95d-402e-bf4d-d8f3823aab9b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:23.566068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2e098-e3dd-434c-b13d-3d1bfe386301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:23.566687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2e098-e3dd-434c-b13d-3d1bfe386301 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:23.566969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2e098-e3dd-434c-b13d-3d1bfe386301 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:23.567241 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2e098-e3dd-434c-b13d-3d1bfe386301 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:23.568353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2e098-e3dd-434c-b13d-3d1bfe386301 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:23.568837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2e098-e3dd-434c-b13d-3d1bfe386301 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:23.570388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2e098-e3dd-434c-b13d-3d1bfe386301 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:23.590225 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 536/1078] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:31:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:23.591715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4303902-08b5-4c52-87d4-1388c5bce90c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:23.679704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:23.704354 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4303902-08b5-4c52-87d4-1388c5bce90c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:23.741053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eebaee-1953-4a14-be52-df540430c81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:23.741737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eebaee-1953-4a14-be52-df540430c81f None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:23.742018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eebaee-1953-4a14-be52-df540430c81f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:23.742289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eebaee-1953-4a14-be52-df540430c81f None admin] PATH_INFO: `/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/e2ecd7d1cce644b0982cc23740537933/roles/039a8882198d4b17911651cf3558be04` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:23.744622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eebaee-1953-4a14-be52-df540430c81f None admin] RBAC: Authorizing `identity:create_grant(project_id=abcccf694e7b4f66a7372e0ede416caf, user_id=e2ecd7d1cce644b0982cc23740537933, role_id=039a8882198d4b17911651cf3558be04)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:23.746083 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eebaee-1953-4a14-be52-df540430c81f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:23.801190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 543/1079] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:31:23 2022] PUT /identity/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/e2ecd7d1cce644b0982cc23740537933/roles/039a8882198d4b17911651cf3558be04 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:31:24.032162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e3942-b5bc-4712-8f66-9d96c2b26190 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:24.032462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e3942-b5bc-4712-8f66-9d96c2b26190 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:24.032643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e3942-b5bc-4712-8f66-9d96c2b26190 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:24.034563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 537/1080] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:24.041018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86638464-120b-4c60-ba5b-5b04468916fd None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:24.041379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86638464-120b-4c60-ba5b-5b04468916fd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:24.041773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86638464-120b-4c60-ba5b-5b04468916fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:24.064107 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-86638464-120b-4c60-ba5b-5b04468916fd None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:24.065307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-86638464-120b-4c60-ba5b-5b04468916fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:24.172739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 544/1081] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:24.195410 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e3942-b5bc-4712-8f66-9d96c2b26190 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:24.199661 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:24.326137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e3942-b5bc-4712-8f66-9d96c2b26190 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:24.360941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0199dd-948a-4165-bf50-cd0916940dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:24.361561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0199dd-948a-4165-bf50-cd0916940dfe None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:24.361946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0199dd-948a-4165-bf50-cd0916940dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:24.362179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0199dd-948a-4165-bf50-cd0916940dfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:24.430949 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0199dd-948a-4165-bf50-cd0916940dfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:24.432396 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0199dd-948a-4165-bf50-cd0916940dfe None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:24.438188 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:24.559284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 538/1082] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:31:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:31:24.560592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86638464-120b-4c60-ba5b-5b04468916fd None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:24.632045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:24.670011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86638464-120b-4c60-ba5b-5b04468916fd None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:24.705700 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7ee90-4afa-46bd-9c8d-6167514568b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:24.706329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7ee90-4afa-46bd-9c8d-6167514568b0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:24.706610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7ee90-4afa-46bd-9c8d-6167514568b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:24.706885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7ee90-4afa-46bd-9c8d-6167514568b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:24.717272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7ee90-4afa-46bd-9c8d-6167514568b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:24.718721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7ee90-4afa-46bd-9c8d-6167514568b0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-ccb7ee90-4afa-46bd-9c8d-6167514568b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:24.733694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:24.734639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:31:24.735548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:24.735548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:24.735548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:24.737867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 545/1083] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:24.741280 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0199dd-948a-4165-bf50-cd0916940dfe None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:24.849898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0199dd-948a-4165-bf50-cd0916940dfe None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:24.852413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:24.900888 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e85169-7d59-4a8a-849e-1bcc641c9b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:24.901446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e85169-7d59-4a8a-849e-1bcc641c9b78 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:24.901749 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e85169-7d59-4a8a-849e-1bcc641c9b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:24.902029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e85169-7d59-4a8a-849e-1bcc641c9b78 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:24.902707 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e85169-7d59-4a8a-849e-1bcc641c9b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:24.903008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e85169-7d59-4a8a-849e-1bcc641c9b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:24.904656 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e85169-7d59-4a8a-849e-1bcc641c9b78 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:24.917566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 539/1084] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:25.533951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ac199-4687-4e57-bd04-4dac5e77acb1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:25.534460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ac199-4687-4e57-bd04-4dac5e77acb1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:25.534460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ac199-4687-4e57-bd04-4dac5e77acb1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:25.535785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 546/1085] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:25.541429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83844e5-71be-4d06-915c-fcedbf83466c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:25.541888 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83844e5-71be-4d06-915c-fcedbf83466c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:25.542007 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83844e5-71be-4d06-915c-fcedbf83466c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:25.560215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a83844e5-71be-4d06-915c-fcedbf83466c None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:25.561408 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a83844e5-71be-4d06-915c-fcedbf83466c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:25.633721 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 540/1086] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:25.658136 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ac199-4687-4e57-bd04-4dac5e77acb1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:25.763622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ac199-4687-4e57-bd04-4dac5e77acb1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:25.780407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:25.797565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12daa5ba-12d2-4053-b484-0b7cc26fc123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:25.798124 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12daa5ba-12d2-4053-b484-0b7cc26fc123 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:25.798403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12daa5ba-12d2-4053-b484-0b7cc26fc123 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:25.798674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12daa5ba-12d2-4053-b484-0b7cc26fc123 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:25.809726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12daa5ba-12d2-4053-b484-0b7cc26fc123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-12daa5ba-12d2-4053-b484-0b7cc26fc123 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:31:25.814482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:25.815558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:25.816443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 547/1087] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:31:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:25.822704 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83844e5-71be-4d06-915c-fcedbf83466c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:25.837585 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9960dc3-2ee5-45fc-b349-089ce6229001 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:25.838253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9960dc3-2ee5-45fc-b349-089ce6229001 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:25.838643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9960dc3-2ee5-45fc-b349-089ce6229001 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:25.840581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 548/1088] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:31:25.845444 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670110c-17c6-426d-954d-ca31e76317be None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:25.845754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670110c-17c6-426d-954d-ca31e76317be None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:25.846030 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670110c-17c6-426d-954d-ca31e76317be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:25.863269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c670110c-17c6-426d-954d-ca31e76317be None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:25.864992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c670110c-17c6-426d-954d-ca31e76317be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:25.915874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:25.922792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83844e5-71be-4d06-915c-fcedbf83466c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:25.932314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 549/1089] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:31:25.955858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670110c-17c6-426d-954d-ca31e76317be None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:25.957416 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ac4c7-253b-4b3d-847f-b867c54fb5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:25.958079 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ac4c7-253b-4b3d-847f-b867c54fb5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:25.958356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ac4c7-253b-4b3d-847f-b867c54fb5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:25.958632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ac4c7-253b-4b3d-847f-b867c54fb5e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:25.959292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ac4c7-253b-4b3d-847f-b867c54fb5e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:25.959593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ac4c7-253b-4b3d-847f-b867c54fb5e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:25.960948 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ac4c7-253b-4b3d-847f-b867c54fb5e7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:25.969342 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 541/1090] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:31:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:25.975050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ac4c7-253b-4b3d-847f-b867c54fb5e7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:26.055108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670110c-17c6-426d-954d-ca31e76317be None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:26.062585 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:26.075908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ac4c7-253b-4b3d-847f-b867c54fb5e7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:26.090678 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504f041-f99f-40cb-ae35-d7df46a5fa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:26.091505 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504f041-f99f-40cb-ae35-d7df46a5fa23 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:26.091974 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504f041-f99f-40cb-ae35-d7df46a5fa23 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:26.092566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504f041-f99f-40cb-ae35-d7df46a5fa23 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:26.102883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504f041-f99f-40cb-ae35-d7df46a5fa23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-0504f041-f99f-40cb-ae35-d7df46a5fa23 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:31:26.107653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:26.109286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:26.110997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 550/1091] 10.222.0.57 () {56 vars in 1120 bytes} [Tue Aug 9 18:31:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:26.113618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccc3fe-7c0d-4226-94c9-18eb5a3ae790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:26.114761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc3fe-7c0d-4226-94c9-18eb5a3ae790 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:26.115043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc3fe-7c0d-4226-94c9-18eb5a3ae790 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:26.115317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc3fe-7c0d-4226-94c9-18eb5a3ae790 None admin] PATH_INFO: `/v3/users/54b21288dd78453a89ca6ffea624db83` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:26.115463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504f041-f99f-40cb-ae35-d7df46a5fa23 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:26.116470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccc3fe-7c0d-4226-94c9-18eb5a3ae790 None admin] RBAC: Authorizing `identity:get_user(user_id=54b21288dd78453a89ca6ffea624db83)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:26.117946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccc3fe-7c0d-4226-94c9-18eb5a3ae790 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:26.119816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 542/1092] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:31:25 2022] GET /identity/v3/users/54b21288dd78453a89ca6ffea624db83 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:26.125249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccc3fe-7c0d-4226-94c9-18eb5a3ae790 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:26.151842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:26.228211 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504f041-f99f-40cb-ae35-d7df46a5fa23 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:26.237650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccc3fe-7c0d-4226-94c9-18eb5a3ae790 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:26.275403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f5cc2-29bf-4503-a50b-12485856d6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:26.276030 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f5cc2-29bf-4503-a50b-12485856d6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:26.276311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f5cc2-29bf-4503-a50b-12485856d6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:26.276581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f5cc2-29bf-4503-a50b-12485856d6b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:26.277260 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f5cc2-29bf-4503-a50b-12485856d6b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:26.277584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f5cc2-29bf-4503-a50b-12485856d6b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:26.278968 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f5cc2-29bf-4503-a50b-12485856d6b6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:26.286975 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35d956-1b7a-41d2-ad68-68bdce84943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:26.287582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35d956-1b7a-41d2-ad68-68bdce84943e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:26.287860 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35d956-1b7a-41d2-ad68-68bdce84943e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:26.288131 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35d956-1b7a-41d2-ad68-68bdce84943e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:26.288763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 551/1093] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 9 18:31:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:26.297103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f5cc2-29bf-4503-a50b-12485856d6b6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:26.329029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:26.366781 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35d956-1b7a-41d2-ad68-68bdce84943e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:26.368251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35d956-1b7a-41d2-ad68-68bdce84943e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:26.393240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:26.405936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f5cc2-29bf-4503-a50b-12485856d6b6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:26.446287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd953da-5b02-4e24-9f96-9d50d4bd08ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:26.446915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd953da-5b02-4e24-9f96-9d50d4bd08ea None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:26.447195 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd953da-5b02-4e24-9f96-9d50d4bd08ea None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:26.447465 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd953da-5b02-4e24-9f96-9d50d4bd08ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:26.458054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd953da-5b02-4e24-9f96-9d50d4bd08ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:26.459561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd953da-5b02-4e24-9f96-9d50d4bd08ea None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-ffd953da-5b02-4e24-9f96-9d50d4bd08ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:26.474690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:26.475660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:31:26.476557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:26.476557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:26.476557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:26.476726 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 543/1094] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:31:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 18:31:26.476798 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 552/1095] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:31:26.482550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35d956-1b7a-41d2-ad68-68bdce84943e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:26.482687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd953da-5b02-4e24-9f96-9d50d4bd08ea None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:26.569602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:26.590790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35d956-1b7a-41d2-ad68-68bdce84943e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:26.591283 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd953da-5b02-4e24-9f96-9d50d4bd08ea None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:26.615856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:26.627808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6e0f5-b59a-4b4e-ae87-bf2be86db7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:26.628426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6e0f5-b59a-4b4e-ae87-bf2be86db7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:26.628829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6e0f5-b59a-4b4e-ae87-bf2be86db7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:26.628995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6e0f5-b59a-4b4e-ae87-bf2be86db7c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:26.629609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6e0f5-b59a-4b4e-ae87-bf2be86db7c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:26.629952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6e0f5-b59a-4b4e-ae87-bf2be86db7c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:26.631370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6e0f5-b59a-4b4e-ae87-bf2be86db7c6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:26.645127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 544/1096] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:26.646483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556adb2-aafc-41d5-b72c-2600cee9d927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:26.647731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556adb2-aafc-41d5-b72c-2600cee9d927 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:26.648318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556adb2-aafc-41d5-b72c-2600cee9d927 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:26.648829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556adb2-aafc-41d5-b72c-2600cee9d927 None admin] PATH_INFO: `/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:26.650874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556adb2-aafc-41d5-b72c-2600cee9d927 None admin] RBAC: Authorizing `identity:get_project(project_id=6d5860cfa51e40f5b1c6fe9425798e9f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:26.653773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556adb2-aafc-41d5-b72c-2600cee9d927 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:26.654980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6e0f5-b59a-4b4e-ae87-bf2be86db7c6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:26.661821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 553/1097] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:31:26 2022] GET /identity/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:26.663871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556adb2-aafc-41d5-b72c-2600cee9d927 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:26.768898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6e0f5-b59a-4b4e-ae87-bf2be86db7c6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:26.782076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556adb2-aafc-41d5-b72c-2600cee9d927 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:26.808000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a02f9-7b68-4e83-aee9-23c6732ae0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:26.808640 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a02f9-7b68-4e83-aee9-23c6732ae0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:26.808922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a02f9-7b68-4e83-aee9-23c6732ae0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:26.809205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a02f9-7b68-4e83-aee9-23c6732ae0c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:26.817314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a02f9-7b68-4e83-aee9-23c6732ae0c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:26.818814 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a02f9-7b68-4e83-aee9-23c6732ae0c9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:26.821896 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80458465-0c1d-418f-a601-191ec0310b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:26.822502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80458465-0c1d-418f-a601-191ec0310b26 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:26.822783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80458465-0c1d-418f-a601-191ec0310b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:26.823063 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80458465-0c1d-418f-a601-191ec0310b26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:26.823790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80458465-0c1d-418f-a601-191ec0310b26 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=6d5860cfa51e40f5b1c6fe9425798e9f, user.id=54b21288dd78453a89ca6ffea624db83) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:26.824106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80458465-0c1d-418f-a601-191ec0310b26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:26.825555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80458465-0c1d-418f-a601-191ec0310b26 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-ee6a02f9-7b68-4e83-aee9-23c6732ae0c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:31:26.831757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:26.832745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:26.834289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:26.834289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:26.834289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:31:26.834289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:31:26.834289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:31:26.834289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:31:26.834289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:31:26.834289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:31:26.834289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 18:31:26.834289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:26.835410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 554/1098] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:31:26 2022] GET /identity/v3/role_assignments?user.id=54b21288dd78453a89ca6ffea624db83&scope.project.id=6d5860cfa51e40f5b1c6fe9425798e9f&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:26.835513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 545/1099] 10.222.0.57 () {56 vars in 1093 bytes} [Tue Aug 9 18:31:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:26.839548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80458465-0c1d-418f-a601-191ec0310b26 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:26.857800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:26.950507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80458465-0c1d-418f-a601-191ec0310b26 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:26.986239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e43f93-0851-4c7a-80c0-f4d130ff4940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:26.986848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e43f93-0851-4c7a-80c0-f4d130ff4940 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:26.987186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e43f93-0851-4c7a-80c0-f4d130ff4940 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:26.987463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e43f93-0851-4c7a-80c0-f4d130ff4940 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:26.988197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e43f93-0851-4c7a-80c0-f4d130ff4940 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:26.988505 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e43f93-0851-4c7a-80c0-f4d130ff4940 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:26.990502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e43f93-0851-4c7a-80c0-f4d130ff4940 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:27.008344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 555/1100] 10.222.0.57 () {56 vars in 1133 bytes} [Tue Aug 9 18:31:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:27.014310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a02f9-7b68-4e83-aee9-23c6732ae0c9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:27.062381 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:27.117116 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a02f9-7b68-4e83-aee9-23c6732ae0c9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:27.151759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0f9d1-f822-4a5e-8c02-edbe5194c689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:27.152360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0f9d1-f822-4a5e-8c02-edbe5194c689 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:27.152641 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0f9d1-f822-4a5e-8c02-edbe5194c689 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:27.152919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0f9d1-f822-4a5e-8c02-edbe5194c689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:27.222778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0f9d1-f822-4a5e-8c02-edbe5194c689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:27.224246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0f9d1-f822-4a5e-8c02-edbe5194c689 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:27.271077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:27.328393 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 546/1101] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:31:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:31:27.334335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e43f93-0851-4c7a-80c0-f4d130ff4940 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:27.403189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:27.434275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e43f93-0851-4c7a-80c0-f4d130ff4940 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:27.468613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d9729-fdae-49a2-86cc-3975c2fe3cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:27.469237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d9729-fdae-49a2-86cc-3975c2fe3cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:27.469540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d9729-fdae-49a2-86cc-3975c2fe3cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:27.469823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d9729-fdae-49a2-86cc-3975c2fe3cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:27.481398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d9729-fdae-49a2-86cc-3975c2fe3cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:27.482892 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d9729-fdae-49a2-86cc-3975c2fe3cb7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-084d9729-fdae-49a2-86cc-3975c2fe3cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:27.497302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:27.498436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:31:27.499516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:27.499516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:31:27.499516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:27.501681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 556/1102] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:27.504315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0f9d1-f822-4a5e-8c02-edbe5194c689 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:27.603617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0f9d1-f822-4a5e-8c02-edbe5194c689 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:27.637869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ec8e2-7101-4885-9ded-440fdd2b33bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:27.638470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ec8e2-7101-4885-9ded-440fdd2b33bc None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:27.638752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ec8e2-7101-4885-9ded-440fdd2b33bc None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:27.639028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ec8e2-7101-4885-9ded-440fdd2b33bc None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:27.639670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ec8e2-7101-4885-9ded-440fdd2b33bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:27.639977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ec8e2-7101-4885-9ded-440fdd2b33bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:27.641337 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ec8e2-7101-4885-9ded-440fdd2b33bc None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:27.653097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 547/1103] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:27.658749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d9729-fdae-49a2-86cc-3975c2fe3cb7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:27.762425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d9729-fdae-49a2-86cc-3975c2fe3cb7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:27.785089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:27.796123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a0810-652f-44ae-96f2-3e66525d2fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:27.796746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a0810-652f-44ae-96f2-3e66525d2fca None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:27.809423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a0810-652f-44ae-96f2-3e66525d2fca None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:27.809423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a0810-652f-44ae-96f2-3e66525d2fca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:27.809423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a0810-652f-44ae-96f2-3e66525d2fca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:27.809423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a0810-652f-44ae-96f2-3e66525d2fca None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-ed2a0810-652f-44ae-96f2-3e66525d2fca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:31:27.816646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:27.818233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:27.819127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:31:27.819127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:31:27.819127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:31:27.819127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:31:27.819127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:31:27.819127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:31:27.819127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:31:27.819127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:27.819127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 557/1104] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:31:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:27.824261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ec8e2-7101-4885-9ded-440fdd2b33bc None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:27.828832 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:27.927349 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ec8e2-7101-4885-9ded-440fdd2b33bc None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:27.961569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feeb38a-0b02-473a-b7dd-b1665d74fc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:27.962183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feeb38a-0b02-473a-b7dd-b1665d74fc29 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:27.962464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feeb38a-0b02-473a-b7dd-b1665d74fc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:27.962735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feeb38a-0b02-473a-b7dd-b1665d74fc29 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:27.963450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feeb38a-0b02-473a-b7dd-b1665d74fc29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:27.963758 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feeb38a-0b02-473a-b7dd-b1665d74fc29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:27.965215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feeb38a-0b02-473a-b7dd-b1665d74fc29 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:27.983360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 548/1105] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:31:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:27.984937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a0810-652f-44ae-96f2-3e66525d2fca None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:28.085013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a0810-652f-44ae-96f2-3e66525d2fca None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:28.118668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128df50-d3be-49a0-bdce-02b02c7e7c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:28.119270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128df50-d3be-49a0-bdce-02b02c7e7c72 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:28.119549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128df50-d3be-49a0-bdce-02b02c7e7c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:28.119820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128df50-d3be-49a0-bdce-02b02c7e7c72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:28.120546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128df50-d3be-49a0-bdce-02b02c7e7c72 None admin] RBAC: Adding query filter params (role.id=039a8882198d4b17911651cf3558be04, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=e2ecd7d1cce644b0982cc23740537933) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:28.120859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128df50-d3be-49a0-bdce-02b02c7e7c72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:28.122332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128df50-d3be-49a0-bdce-02b02c7e7c72 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:28.128364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 558/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:31:27 2022] GET /identity/v3/role_assignments?user.id=e2ecd7d1cce644b0982cc23740537933&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=039a8882198d4b17911651cf3558be04 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:28.871957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05a3ce-6783-49d8-a301-beec222820f5 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:28.872588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05a3ce-6783-49d8-a301-beec222820f5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:28.872588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05a3ce-6783-49d8-a301-beec222820f5 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:28.873846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 549/1107] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:28.879422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5bd5c-a81a-472d-83c4-0a6c3970567d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:28.879928 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5bd5c-a81a-472d-83c4-0a6c3970567d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:28.880036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5bd5c-a81a-472d-83c4-0a6c3970567d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:28.900281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-abc5bd5c-a81a-472d-83c4-0a6c3970567d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:28.901485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-abc5bd5c-a81a-472d-83c4-0a6c3970567d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:28.945993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:28.971406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 559/1108] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:28.996222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05a3ce-6783-49d8-a301-beec222820f5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:29.065286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:29.105693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05a3ce-6783-49d8-a301-beec222820f5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:29.139573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554e8c5-b5fe-4718-be51-87f3e983cd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:29.140114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554e8c5-b5fe-4718-be51-87f3e983cd02 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:29.140445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554e8c5-b5fe-4718-be51-87f3e983cd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:29.140668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554e8c5-b5fe-4718-be51-87f3e983cd02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:29.141811 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554e8c5-b5fe-4718-be51-87f3e983cd02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:29.143230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554e8c5-b5fe-4718-be51-87f3e983cd02 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:29.144782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 550/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:29.150585 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5bd5c-a81a-472d-83c4-0a6c3970567d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:29.247801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5bd5c-a81a-472d-83c4-0a6c3970567d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:29.282347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db7fb8-3aca-45ee-b052-3f74bd53e867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:29.282951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db7fb8-3aca-45ee-b052-3f74bd53e867 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:29.283232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db7fb8-3aca-45ee-b052-3f74bd53e867 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:29.283505 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db7fb8-3aca-45ee-b052-3f74bd53e867 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:29.284141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db7fb8-3aca-45ee-b052-3f74bd53e867 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:29.285606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db7fb8-3aca-45ee-b052-3f74bd53e867 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:29.388524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 560/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:31:29 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:31:31.448404 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8daab12-7a6f-4391-8a43-31fd84c7fcea None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:31.448951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8daab12-7a6f-4391-8a43-31fd84c7fcea None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:31.448951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8daab12-7a6f-4391-8a43-31fd84c7fcea None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:31.450302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 551/1111] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:31.455703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31754bd-7fe8-49cf-ac34-b2284e58c81f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:31.456137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31754bd-7fe8-49cf-ac34-b2284e58c81f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:31.456237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31754bd-7fe8-49cf-ac34-b2284e58c81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:31.474584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e31754bd-7fe8-49cf-ac34-b2284e58c81f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:31.475763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e31754bd-7fe8-49cf-ac34-b2284e58c81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:31.502018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:31.542572 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 561/1112] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:31.566600 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8daab12-7a6f-4391-8a43-31fd84c7fcea None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:31.665536 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8daab12-7a6f-4391-8a43-31fd84c7fcea None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:31.695471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:31.700318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea470a8-9f45-4504-9e11-9bf224526bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:31.700942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea470a8-9f45-4504-9e11-9bf224526bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:31.701222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea470a8-9f45-4504-9e11-9bf224526bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:31.701496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea470a8-9f45-4504-9e11-9bf224526bc6 None admin] PATH_INFO: `/v3/roles/1d1b5b96118040f9a4fd019bbcd93071` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:31.703318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea470a8-9f45-4504-9e11-9bf224526bc6 None admin] RBAC: Authorizing `identity:get_role(role_id=1d1b5b96118040f9a4fd019bbcd93071)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:31.704675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea470a8-9f45-4504-9e11-9bf224526bc6 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:31.706030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 552/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:31:31 2022] GET /identity/v3/roles/1d1b5b96118040f9a4fd019bbcd93071 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:31.711863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31754bd-7fe8-49cf-ac34-b2284e58c81f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:31.810642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31754bd-7fe8-49cf-ac34-b2284e58c81f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:31.843697 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc9904-f59a-455a-bf18-fb791af60abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:31.844314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc9904-f59a-455a-bf18-fb791af60abd None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:31.844592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc9904-f59a-455a-bf18-fb791af60abd None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:31.844862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc9904-f59a-455a-bf18-fb791af60abd None admin] PATH_INFO: `/v3/users/56f41dc8d06645dd93e640848997a8f8` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:31.860622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc9904-f59a-455a-bf18-fb791af60abd None admin] RBAC: Authorizing `identity:get_user(user_id=56f41dc8d06645dd93e640848997a8f8)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:31.862169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc9904-f59a-455a-bf18-fb791af60abd None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:31.863794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 562/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:31:31 2022] GET /identity/v3/users/56f41dc8d06645dd93e640848997a8f8 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:31.869813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea470a8-9f45-4504-9e11-9bf224526bc6 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:31.965968 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea470a8-9f45-4504-9e11-9bf224526bc6 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:31.997973 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea117e-5ba4-4fd3-aa25-c6a754229816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:31.998554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea117e-5ba4-4fd3-aa25-c6a754229816 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:31.998829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea117e-5ba4-4fd3-aa25-c6a754229816 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:31.999096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea117e-5ba4-4fd3-aa25-c6a754229816 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:32.067853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:32.067853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea117e-5ba4-4fd3-aa25-c6a754229816 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:32.067853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea117e-5ba4-4fd3-aa25-c6a754229816 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:32.163964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 553/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:31:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:31:32.170082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc9904-f59a-455a-bf18-fb791af60abd None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:32.192175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:32.268082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc9904-f59a-455a-bf18-fb791af60abd None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:32.305026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06cf13-7885-49a3-b471-d25a12f401f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:32.305687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06cf13-7885-49a3-b471-d25a12f401f0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:32.305964 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06cf13-7885-49a3-b471-d25a12f401f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:32.306235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06cf13-7885-49a3-b471-d25a12f401f0 None admin] PATH_INFO: `/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:32.307379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06cf13-7885-49a3-b471-d25a12f401f0 None admin] RBAC: Authorizing `identity:get_project(project_id=6d5860cfa51e40f5b1c6fe9425798e9f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:32.308802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06cf13-7885-49a3-b471-d25a12f401f0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:32.310463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 563/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:31:32 2022] GET /identity/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:32.316831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea117e-5ba4-4fd3-aa25-c6a754229816 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:32.421302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:32.423967 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea117e-5ba4-4fd3-aa25-c6a754229816 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:32.456693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09f656-b756-4c80-a620-1f26dc1eaa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:32.457408 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09f656-b756-4c80-a620-1f26dc1eaa4c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:32.457600 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09f656-b756-4c80-a620-1f26dc1eaa4c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:32.458452 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09f656-b756-4c80-a620-1f26dc1eaa4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:32.459225 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09f656-b756-4c80-a620-1f26dc1eaa4c None admin] RBAC: Adding query filter params (role.id=1d1b5b96118040f9a4fd019bbcd93071, scope.project.id=6d5860cfa51e40f5b1c6fe9425798e9f, user.id=56f41dc8d06645dd93e640848997a8f8) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:32.459536 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09f656-b756-4c80-a620-1f26dc1eaa4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:32.460982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09f656-b756-4c80-a620-1f26dc1eaa4c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:32.467103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 554/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:31:32 2022] GET /identity/v3/role_assignments?user.id=56f41dc8d06645dd93e640848997a8f8&scope.project.id=6d5860cfa51e40f5b1c6fe9425798e9f&role.id=1d1b5b96118040f9a4fd019bbcd93071 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:34.460002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5495197-4dd8-4b64-9c6f-c5fdcd923d2c None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:34.460525 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5495197-4dd8-4b64-9c6f-c5fdcd923d2c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:34.460525 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5495197-4dd8-4b64-9c6f-c5fdcd923d2c None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:34.461925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 564/1118] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:34.467176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d2bb5-2d59-42c9-8d7f-123efe05bc0a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:34.467613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d2bb5-2d59-42c9-8d7f-123efe05bc0a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:34.467724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d2bb5-2d59-42c9-8d7f-123efe05bc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:34.486289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-fa8d2bb5-2d59-42c9-8d7f-123efe05bc0a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:34.487477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fa8d2bb5-2d59-42c9-8d7f-123efe05bc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:34.553398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 555/1119] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:34.576239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5495197-4dd8-4b64-9c6f-c5fdcd923d2c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:34.667883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:34.680535 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5495197-4dd8-4b64-9c6f-c5fdcd923d2c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:34.721742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb811724-9049-4f98-b7e3-682f6f6bed69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:34.722783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb811724-9049-4f98-b7e3-682f6f6bed69 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:34.723621 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb811724-9049-4f98-b7e3-682f6f6bed69 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:34.724106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb811724-9049-4f98-b7e3-682f6f6bed69 None admin] PATH_INFO: `/v3/roles/1d1b5b96118040f9a4fd019bbcd93071` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:34.726517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb811724-9049-4f98-b7e3-682f6f6bed69 None admin] RBAC: Authorizing `identity:get_role(role_id=1d1b5b96118040f9a4fd019bbcd93071)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:34.728889 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb811724-9049-4f98-b7e3-682f6f6bed69 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:34.730563 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 565/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:31:34 2022] GET /identity/v3/roles/1d1b5b96118040f9a4fd019bbcd93071 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:34.736472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d2bb5-2d59-42c9-8d7f-123efe05bc0a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:34.828258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:34.834713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d2bb5-2d59-42c9-8d7f-123efe05bc0a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:34.866865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc6a3b-88dd-448e-92b8-8b814a1d67b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:34.867532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc6a3b-88dd-448e-92b8-8b814a1d67b8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:34.868012 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc6a3b-88dd-448e-92b8-8b814a1d67b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:34.868444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc6a3b-88dd-448e-92b8-8b814a1d67b8 None admin] PATH_INFO: `/v3/users/56f41dc8d06645dd93e640848997a8f8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:34.869908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc6a3b-88dd-448e-92b8-8b814a1d67b8 None admin] RBAC: Authorizing `identity:get_user(user_id=56f41dc8d06645dd93e640848997a8f8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:34.871329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc6a3b-88dd-448e-92b8-8b814a1d67b8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:34.872885 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 556/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:31:34 2022] GET /identity/v3/users/56f41dc8d06645dd93e640848997a8f8 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:34.878754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb811724-9049-4f98-b7e3-682f6f6bed69 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:34.984366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb811724-9049-4f98-b7e3-682f6f6bed69 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:35.017335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031099d0-6f03-4477-8736-ad0584e0bd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:35.017980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031099d0-6f03-4477-8736-ad0584e0bd67 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:35.018259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031099d0-6f03-4477-8736-ad0584e0bd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:35.018527 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031099d0-6f03-4477-8736-ad0584e0bd67 None admin] PATH_INFO: `/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:35.019630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031099d0-6f03-4477-8736-ad0584e0bd67 None admin] RBAC: Authorizing `identity:get_project(project_id=6d5860cfa51e40f5b1c6fe9425798e9f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:35.021729 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031099d0-6f03-4477-8736-ad0584e0bd67 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:35.023349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 566/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:31:34 2022] GET /identity/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:35.029359 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc6a3b-88dd-448e-92b8-8b814a1d67b8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:35.127511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc6a3b-88dd-448e-92b8-8b814a1d67b8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:35.163083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725869d-1e9c-4a80-95f7-873cc97da055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:35.163736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725869d-1e9c-4a80-95f7-873cc97da055 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:35.164019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725869d-1e9c-4a80-95f7-873cc97da055 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:35.164534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725869d-1e9c-4a80-95f7-873cc97da055 None admin] PATH_INFO: `/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f/users/56f41dc8d06645dd93e640848997a8f8/roles/1d1b5b96118040f9a4fd019bbcd93071` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:35.166505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725869d-1e9c-4a80-95f7-873cc97da055 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d5860cfa51e40f5b1c6fe9425798e9f, user_id=56f41dc8d06645dd93e640848997a8f8, role_id=1d1b5b96118040f9a4fd019bbcd93071)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:35.167930 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725869d-1e9c-4a80-95f7-873cc97da055 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:35.239977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 557/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:31:35 2022] PUT /identity/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f/users/56f41dc8d06645dd93e640848997a8f8/roles/1d1b5b96118040f9a4fd019bbcd93071 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:31:37.294737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b5439-613f-4ce3-bae3-65ff5d08b8e4 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:37.295213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b5439-613f-4ce3-bae3-65ff5d08b8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:37.295317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b5439-613f-4ce3-bae3-65ff5d08b8e4 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:37.296682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 567/1124] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:37.301876 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe252b9-1b34-4069-acb4-eced6888c2ee None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:37.302302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe252b9-1b34-4069-acb4-eced6888c2ee None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:37.302411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe252b9-1b34-4069-acb4-eced6888c2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:37.320448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-dbe252b9-1b34-4069-acb4-eced6888c2ee None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:37.321637 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dbe252b9-1b34-4069-acb4-eced6888c2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:37.337966 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:37.388063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 558/1125] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:37.411720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b5439-613f-4ce3-bae3-65ff5d08b8e4 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:37.436871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:37.512460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b5439-613f-4ce3-bae3-65ff5d08b8e4 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:37.545578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e623016-1c0a-4db7-be9e-3a0dd3a4f4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:37.546142 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e623016-1c0a-4db7-be9e-3a0dd3a4f4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:37.546424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e623016-1c0a-4db7-be9e-3a0dd3a4f4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:37.546703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e623016-1c0a-4db7-be9e-3a0dd3a4f4c2 None admin] PATH_INFO: `/v3/roles/1d1b5b96118040f9a4fd019bbcd93071` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:37.548139 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e623016-1c0a-4db7-be9e-3a0dd3a4f4c2 None admin] RBAC: Authorizing `identity:get_role(role_id=1d1b5b96118040f9a4fd019bbcd93071)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:37.549504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e623016-1c0a-4db7-be9e-3a0dd3a4f4c2 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:37.550837 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 568/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:31:37 2022] GET /identity/v3/roles/1d1b5b96118040f9a4fd019bbcd93071 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:37.556932 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe252b9-1b34-4069-acb4-eced6888c2ee None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:37.658178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe252b9-1b34-4069-acb4-eced6888c2ee None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:37.691718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87894c0c-a498-4241-94d0-c309137d21ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:37.692872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87894c0c-a498-4241-94d0-c309137d21ec None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:37.693157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87894c0c-a498-4241-94d0-c309137d21ec None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:37.693431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87894c0c-a498-4241-94d0-c309137d21ec None admin] PATH_INFO: `/v3/users/56f41dc8d06645dd93e640848997a8f8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:37.694581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87894c0c-a498-4241-94d0-c309137d21ec None admin] RBAC: Authorizing `identity:get_user(user_id=56f41dc8d06645dd93e640848997a8f8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:37.696006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87894c0c-a498-4241-94d0-c309137d21ec None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:37.697591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 559/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:31:37 2022] GET /identity/v3/users/56f41dc8d06645dd93e640848997a8f8 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:37.703756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e623016-1c0a-4db7-be9e-3a0dd3a4f4c2 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:37.794512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:37.804273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e623016-1c0a-4db7-be9e-3a0dd3a4f4c2 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:37.837148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b4b3d-6512-447c-81b6-fe9c1d49f6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:37.837768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b4b3d-6512-447c-81b6-fe9c1d49f6eb None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:37.838051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b4b3d-6512-447c-81b6-fe9c1d49f6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:37.838323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b4b3d-6512-447c-81b6-fe9c1d49f6eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:37.903683 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b4b3d-6512-447c-81b6-fe9c1d49f6eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:37.905148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b4b3d-6512-447c-81b6-fe9c1d49f6eb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:37.992895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:38.002314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 569/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:31:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:31:38.008935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87894c0c-a498-4241-94d0-c309137d21ec None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:38.030472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:38.106173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87894c0c-a498-4241-94d0-c309137d21ec None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:38.139556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5939c1-32e2-48ba-b24f-c28e56789bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:38.140168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5939c1-32e2-48ba-b24f-c28e56789bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:38.140447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5939c1-32e2-48ba-b24f-c28e56789bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:38.140719 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5939c1-32e2-48ba-b24f-c28e56789bc2 None admin] PATH_INFO: `/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:38.141874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5939c1-32e2-48ba-b24f-c28e56789bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=6d5860cfa51e40f5b1c6fe9425798e9f)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:38.143297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5939c1-32e2-48ba-b24f-c28e56789bc2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:38.144897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 560/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:31:38 2022] GET /identity/v3/projects/6d5860cfa51e40f5b1c6fe9425798e9f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:38.151167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b4b3d-6512-447c-81b6-fe9c1d49f6eb None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:38.247224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b4b3d-6512-447c-81b6-fe9c1d49f6eb None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:38.280771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8684a-86a1-4234-b5e4-53f5b400d58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:38.281378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8684a-86a1-4234-b5e4-53f5b400d58f None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:38.281681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8684a-86a1-4234-b5e4-53f5b400d58f None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:38.281953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8684a-86a1-4234-b5e4-53f5b400d58f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:38.282679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8684a-86a1-4234-b5e4-53f5b400d58f None admin] RBAC: Adding query filter params (role.id=1d1b5b96118040f9a4fd019bbcd93071, scope.project.id=6d5860cfa51e40f5b1c6fe9425798e9f, user.id=56f41dc8d06645dd93e640848997a8f8) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:38.282989 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8684a-86a1-4234-b5e4-53f5b400d58f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:38.284413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8684a-86a1-4234-b5e4-53f5b400d58f None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:38.290985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 570/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:31:38 2022] GET /identity/v3/role_assignments?user.id=56f41dc8d06645dd93e640848997a8f8&scope.project.id=6d5860cfa51e40f5b1c6fe9425798e9f&role.id=1d1b5b96118040f9a4fd019bbcd93071 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:40.326787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4200790-c95b-479c-bf8c-c97c816e917c None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:40.327331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4200790-c95b-479c-bf8c-c97c816e917c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:40.327331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4200790-c95b-479c-bf8c-c97c816e917c None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:40.328672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 561/1131] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:40.334196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9e65e-e881-4d2c-a8f8-789e7619be2e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:40.334648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9e65e-e881-4d2c-a8f8-789e7619be2e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:40.334648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9e65e-e881-4d2c-a8f8-789e7619be2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:40.352445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0ea9e65e-e881-4d2c-a8f8-789e7619be2e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:40.353675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0ea9e65e-e881-4d2c-a8f8-789e7619be2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:40.403937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:40.421724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 571/1132] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:40.445495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4200790-c95b-479c-bf8c-c97c816e917c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:40.530179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:40.542750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4200790-c95b-479c-bf8c-c97c816e917c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:40.576230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb02321-249c-427d-8ac5-33c6d9acb9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:40.576842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb02321-249c-427d-8ac5-33c6d9acb9fd None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:40.577117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb02321-249c-427d-8ac5-33c6d9acb9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:40.577385 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb02321-249c-427d-8ac5-33c6d9acb9fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:40.578517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb02321-249c-427d-8ac5-33c6d9acb9fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:40.579927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb02321-249c-427d-8ac5-33c6d9acb9fd None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:40.581668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 562/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:40.587524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9e65e-e881-4d2c-a8f8-789e7619be2e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:40.714028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9e65e-e881-4d2c-a8f8-789e7619be2e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:40.747767 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f348b0b-3318-4e34-b4eb-392f4f466e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:40.748363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f348b0b-3318-4e34-b4eb-392f4f466e7e None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:40.748639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f348b0b-3318-4e34-b4eb-392f4f466e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:40.748906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f348b0b-3318-4e34-b4eb-392f4f466e7e None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:40.749564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f348b0b-3318-4e34-b4eb-392f4f466e7e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:40.750991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f348b0b-3318-4e34-b4eb-392f4f466e7e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:40.846486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 572/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:31:40 2022] POST /identity/v3/projects => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:31:42.884022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb2448-52bf-4669-85d5-f1c54fe410dc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:42.884557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb2448-52bf-4669-85d5-f1c54fe410dc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:42.884557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb2448-52bf-4669-85d5-f1c54fe410dc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:42.885908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 563/1135] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:42.891129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a17ac0-b922-403d-bfce-416039060f7d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:42.891531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a17ac0-b922-403d-bfce-416039060f7d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:42.891678 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a17ac0-b922-403d-bfce-416039060f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:42.910316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d4a17ac0-b922-403d-bfce-416039060f7d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:42.911510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d4a17ac0-b922-403d-bfce-416039060f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:42.940832 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:42.978317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 573/1136] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:43.002915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb2448-52bf-4669-85d5-f1c54fe410dc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:43.102177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb2448-52bf-4669-85d5-f1c54fe410dc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:43.135484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917077f-7271-4a9e-8d7b-87e7719e5e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:43.136035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917077f-7271-4a9e-8d7b-87e7719e5e87 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:43.136315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917077f-7271-4a9e-8d7b-87e7719e5e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:43.136587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917077f-7271-4a9e-8d7b-87e7719e5e87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:43.137716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917077f-7271-4a9e-8d7b-87e7719e5e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:43.139135 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917077f-7271-4a9e-8d7b-87e7719e5e87 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:43.140717 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 564/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:31:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:43.146805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a17ac0-b922-403d-bfce-416039060f7d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:43.243804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a17ac0-b922-403d-bfce-416039060f7d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:43.277107 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ec8ab-9e36-4233-b4e8-cf980f533359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:43.277759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ec8ab-9e36-4233-b4e8-cf980f533359 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:43.278043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ec8ab-9e36-4233-b4e8-cf980f533359 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:43.278318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ec8ab-9e36-4233-b4e8-cf980f533359 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:43.278973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ec8ab-9e36-4233-b4e8-cf980f533359 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:43.280400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ec8ab-9e36-4233-b4e8-cf980f533359 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:43.359134 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 574/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:31:43 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:31:45.422414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fd8e7-4f68-49bc-9c6d-decd715ff490 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:45.422971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fd8e7-4f68-49bc-9c6d-decd715ff490 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:45.422971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fd8e7-4f68-49bc-9c6d-decd715ff490 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:45.424272 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 565/1139] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:45.429828 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975976e-6c22-4aed-af27-039dcb6dd4a5 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:45.430333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975976e-6c22-4aed-af27-039dcb6dd4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:45.430440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975976e-6c22-4aed-af27-039dcb6dd4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:45.449170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8975976e-6c22-4aed-af27-039dcb6dd4a5 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:45.450383 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8975976e-6c22-4aed-af27-039dcb6dd4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:45.467655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:45.516924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 575/1140] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:45.540941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fd8e7-4f68-49bc-9c6d-decd715ff490 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:45.565881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:45.640451 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fd8e7-4f68-49bc-9c6d-decd715ff490 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:45.674270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaec98b-982d-4837-a244-473d06d8549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:45.674886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaec98b-982d-4837-a244-473d06d8549b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:45.675168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaec98b-982d-4837-a244-473d06d8549b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:45.675443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaec98b-982d-4837-a244-473d06d8549b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:45.685701 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaec98b-982d-4837-a244-473d06d8549b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-eaaec98b-982d-4837-a244-473d06d8549b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:31:45.690332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:45.692031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:45.693296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 566/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:31:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:45.698104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975976e-6c22-4aed-af27-039dcb6dd4a5 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:45.796550 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975976e-6c22-4aed-af27-039dcb6dd4a5 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:45.829931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54353e1-ccc2-495b-889b-aa44a410defc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:45.830522 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54353e1-ccc2-495b-889b-aa44a410defc None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:45.830792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54353e1-ccc2-495b-889b-aa44a410defc None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:45.831059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54353e1-ccc2-495b-889b-aa44a410defc None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:45.831709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54353e1-ccc2-495b-889b-aa44a410defc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:45.832012 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54353e1-ccc2-495b-889b-aa44a410defc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:45.833979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54353e1-ccc2-495b-889b-aa44a410defc None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:45.842079 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 576/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:31:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:45.848142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaec98b-982d-4837-a244-473d06d8549b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:45.930035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:45.945648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaec98b-982d-4837-a244-473d06d8549b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:45.978907 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c3c9d-1ef2-4f75-844b-10c6783e7276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:45.979520 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c3c9d-1ef2-4f75-844b-10c6783e7276 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:45.979802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c3c9d-1ef2-4f75-844b-10c6783e7276 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:45.980075 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c3c9d-1ef2-4f75-844b-10c6783e7276 None admin] PATH_INFO: `/v3/users/94427a9d17474af6a8f21162bb03709d` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:45.993416 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c3c9d-1ef2-4f75-844b-10c6783e7276 None admin] RBAC: Authorizing `identity:get_user(user_id=94427a9d17474af6a8f21162bb03709d)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:45.994880 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c3c9d-1ef2-4f75-844b-10c6783e7276 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:45.996466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 567/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:31:45 2022] GET /identity/v3/users/94427a9d17474af6a8f21162bb03709d => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:46.002392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54353e1-ccc2-495b-889b-aa44a410defc None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:46.021666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:46.118355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54353e1-ccc2-495b-889b-aa44a410defc None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:46.152202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4a788-38a1-4387-b7ad-513f45b445e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:46.152809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4a788-38a1-4387-b7ad-513f45b445e9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:46.153097 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4a788-38a1-4387-b7ad-513f45b445e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:46.153397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4a788-38a1-4387-b7ad-513f45b445e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:46.219038 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4a788-38a1-4387-b7ad-513f45b445e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:46.220490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4a788-38a1-4387-b7ad-513f45b445e9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:46.236743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:46.318943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 577/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:31:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:31:46.325044 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c3c9d-1ef2-4f75-844b-10c6783e7276 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:46.432978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c3c9d-1ef2-4f75-844b-10c6783e7276 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:46.467169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b531435-7e67-40b2-b03f-aea309bd69b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:46.467794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b531435-7e67-40b2-b03f-aea309bd69b0 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:46.468074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b531435-7e67-40b2-b03f-aea309bd69b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:46.468345 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b531435-7e67-40b2-b03f-aea309bd69b0 None admin] PATH_INFO: `/v3/projects/12cf9a3762c64ab2990640df89b01406` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:46.469455 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b531435-7e67-40b2-b03f-aea309bd69b0 None admin] RBAC: Authorizing `identity:get_project(project_id=12cf9a3762c64ab2990640df89b01406)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:46.471043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b531435-7e67-40b2-b03f-aea309bd69b0 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:46.473241 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 568/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:31:46 2022] GET /identity/v3/projects/12cf9a3762c64ab2990640df89b01406 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:46.479564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4a788-38a1-4387-b7ad-513f45b445e9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:46.578092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4a788-38a1-4387-b7ad-513f45b445e9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:46.593377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:46.611372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ffa7a-5e50-4863-a344-17c57630638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:46.617934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ffa7a-5e50-4863-a344-17c57630638a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:46.617934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ffa7a-5e50-4863-a344-17c57630638a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:46.617934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ffa7a-5e50-4863-a344-17c57630638a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:46.617934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ffa7a-5e50-4863-a344-17c57630638a None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=12cf9a3762c64ab2990640df89b01406, user.id=94427a9d17474af6a8f21162bb03709d) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:46.617934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ffa7a-5e50-4863-a344-17c57630638a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:46.617934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ffa7a-5e50-4863-a344-17c57630638a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:46.620730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 578/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:31:46 2022] GET /identity/v3/role_assignments?user.id=94427a9d17474af6a8f21162bb03709d&scope.project.id=12cf9a3762c64ab2990640df89b01406&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:48.621972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3931ded-fb80-4753-adaa-c6797cf747dc None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:48.622505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3931ded-fb80-4753-adaa-c6797cf747dc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:48.622505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3931ded-fb80-4753-adaa-c6797cf747dc None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:48.623847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 569/1147] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:48.629303 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee862d8-3236-47b3-91f5-af3866b4599a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:48.629748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee862d8-3236-47b3-91f5-af3866b4599a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:48.629935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee862d8-3236-47b3-91f5-af3866b4599a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:48.648332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bee862d8-3236-47b3-91f5-af3866b4599a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:48.649561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bee862d8-3236-47b3-91f5-af3866b4599a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:48.723336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 579/1148] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:48.746676 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3931ded-fb80-4753-adaa-c6797cf747dc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:48.759681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:48.847156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3931ded-fb80-4753-adaa-c6797cf747dc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:48.885492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981bd99-2eef-492f-a7d7-baa3ea7707bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:48.886058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981bd99-2eef-492f-a7d7-baa3ea7707bf None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:48.886337 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981bd99-2eef-492f-a7d7-baa3ea7707bf None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:48.886609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981bd99-2eef-492f-a7d7-baa3ea7707bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:48.897666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981bd99-2eef-492f-a7d7-baa3ea7707bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-8981bd99-2eef-492f-a7d7-baa3ea7707bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:31:48.902499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:48.903574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:48.904414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:48.904414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:48.904414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:48.904414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:48.904414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:31:48.904414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:31:48.904414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:31:48.904414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:31:48.904414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:31:48.904414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:31:48.904414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:31:48.904414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:48.905678 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 570/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:31:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:48.910016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee862d8-3236-47b3-91f5-af3866b4599a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:49.007449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee862d8-3236-47b3-91f5-af3866b4599a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:49.019225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:49.042727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf09c9-598d-4e53-9e4f-caaaafd03d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:49.043326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf09c9-598d-4e53-9e4f-caaaafd03d19 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:49.043601 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf09c9-598d-4e53-9e4f-caaaafd03d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:49.043869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf09c9-598d-4e53-9e4f-caaaafd03d19 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:49.044519 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf09c9-598d-4e53-9e4f-caaaafd03d19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:49.044825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf09c9-598d-4e53-9e4f-caaaafd03d19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:49.046208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf09c9-598d-4e53-9e4f-caaaafd03d19 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:49.054934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 580/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:31:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:49.060886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981bd99-2eef-492f-a7d7-baa3ea7707bf None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:49.132048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:49.162392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981bd99-2eef-492f-a7d7-baa3ea7707bf None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:49.198135 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117781d0-7352-4343-af76-14970e30d6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:49.198781 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117781d0-7352-4343-af76-14970e30d6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:49.199062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117781d0-7352-4343-af76-14970e30d6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:49.199337 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117781d0-7352-4343-af76-14970e30d6c9 None admin] PATH_INFO: `/v3/users/94427a9d17474af6a8f21162bb03709d` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:49.200621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117781d0-7352-4343-af76-14970e30d6c9 None admin] RBAC: Authorizing `identity:get_user(user_id=94427a9d17474af6a8f21162bb03709d)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:49.202100 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117781d0-7352-4343-af76-14970e30d6c9 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:49.203853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 571/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:31:49 2022] GET /identity/v3/users/94427a9d17474af6a8f21162bb03709d => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:49.209628 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf09c9-598d-4e53-9e4f-caaaafd03d19 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:49.321353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf09c9-598d-4e53-9e4f-caaaafd03d19 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:49.354952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c19d1-b7dd-4c2e-81a5-bd0805938cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:49.355576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c19d1-b7dd-4c2e-81a5-bd0805938cad None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:49.355858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c19d1-b7dd-4c2e-81a5-bd0805938cad None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:49.356132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c19d1-b7dd-4c2e-81a5-bd0805938cad None admin] PATH_INFO: `/v3/projects/12cf9a3762c64ab2990640df89b01406` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:49.357254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c19d1-b7dd-4c2e-81a5-bd0805938cad None admin] RBAC: Authorizing `identity:get_project(project_id=12cf9a3762c64ab2990640df89b01406)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:49.358755 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c19d1-b7dd-4c2e-81a5-bd0805938cad None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:49.360395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 581/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:31:49 2022] GET /identity/v3/projects/12cf9a3762c64ab2990640df89b01406 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:31:49.366933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117781d0-7352-4343-af76-14970e30d6c9 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:49.465482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117781d0-7352-4343-af76-14970e30d6c9 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:49.499472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd51cb8-2732-49c3-8dfc-75cb402587a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:49.500196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd51cb8-2732-49c3-8dfc-75cb402587a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:49.500480 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd51cb8-2732-49c3-8dfc-75cb402587a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:49.500752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd51cb8-2732-49c3-8dfc-75cb402587a1 None admin] PATH_INFO: `/v3/projects/12cf9a3762c64ab2990640df89b01406/users/94427a9d17474af6a8f21162bb03709d/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:49.502649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd51cb8-2732-49c3-8dfc-75cb402587a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=12cf9a3762c64ab2990640df89b01406, user_id=94427a9d17474af6a8f21162bb03709d, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:49.504101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd51cb8-2732-49c3-8dfc-75cb402587a1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:49.571901 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:49.577395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 572/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:31:49 2022] PUT /identity/v3/projects/12cf9a3762c64ab2990640df89b01406/users/94427a9d17474af6a8f21162bb03709d/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:31:51.534038 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188b615-15fa-4031-b393-c5c8cac3ca50 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:51.534546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188b615-15fa-4031-b393-c5c8cac3ca50 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:51.534546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188b615-15fa-4031-b393-c5c8cac3ca50 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:51.535855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 582/1154] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:51.541114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e1a87-6081-40c8-9d71-dfa03a9c0cf4 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:51.541516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e1a87-6081-40c8-9d71-dfa03a9c0cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:51.541705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e1a87-6081-40c8-9d71-dfa03a9c0cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:51.560128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4a4e1a87-6081-40c8-9d71-dfa03a9c0cf4 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:51.561306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4a4e1a87-6081-40c8-9d71-dfa03a9c0cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:51.629551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 573/1155] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:51.653340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188b615-15fa-4031-b393-c5c8cac3ca50 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:51.716539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:51.781116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188b615-15fa-4031-b393-c5c8cac3ca50 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:51.814808 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213f59c-0075-446a-8a2e-e37be13027c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:51.815433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213f59c-0075-446a-8a2e-e37be13027c3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:51.815710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213f59c-0075-446a-8a2e-e37be13027c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:51.815985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213f59c-0075-446a-8a2e-e37be13027c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:51.826532 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213f59c-0075-446a-8a2e-e37be13027c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-e213f59c-0075-446a-8a2e-e37be13027c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:31:51.831243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:51.832932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:31:51.833839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 583/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:31:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:51.839154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e1a87-6081-40c8-9d71-dfa03a9c0cf4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:51.934360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e1a87-6081-40c8-9d71-dfa03a9c0cf4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:51.960524 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:51.967955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412e1d3-3d4f-4fab-b371-7633a69a8dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:51.968555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412e1d3-3d4f-4fab-b371-7633a69a8dfb None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:51.968835 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412e1d3-3d4f-4fab-b371-7633a69a8dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:51.969156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412e1d3-3d4f-4fab-b371-7633a69a8dfb None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:51.969888 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412e1d3-3d4f-4fab-b371-7633a69a8dfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:51.975877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412e1d3-3d4f-4fab-b371-7633a69a8dfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:51.975877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412e1d3-3d4f-4fab-b371-7633a69a8dfb None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:51.979507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 574/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:31:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:51.985307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213f59c-0075-446a-8a2e-e37be13027c3 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:52.107223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:52.110170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213f59c-0075-446a-8a2e-e37be13027c3 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:52.144425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af1ebb-6ed7-4fd6-b549-b32e9ade0594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:52.145040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1ebb-6ed7-4fd6-b549-b32e9ade0594 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:52.145320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1ebb-6ed7-4fd6-b549-b32e9ade0594 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:52.145618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1ebb-6ed7-4fd6-b549-b32e9ade0594 None admin] PATH_INFO: `/v3/users/94427a9d17474af6a8f21162bb03709d` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:52.146938 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af1ebb-6ed7-4fd6-b549-b32e9ade0594 None admin] RBAC: Authorizing `identity:get_user(user_id=94427a9d17474af6a8f21162bb03709d)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:52.148381 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af1ebb-6ed7-4fd6-b549-b32e9ade0594 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:52.150002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 584/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:31:51 2022] GET /identity/v3/users/94427a9d17474af6a8f21162bb03709d => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:52.155899 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412e1d3-3d4f-4fab-b371-7633a69a8dfb None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:52.255958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412e1d3-3d4f-4fab-b371-7633a69a8dfb None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:52.289783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188bdc0-28cc-4422-807a-189a82015898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:52.290389 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188bdc0-28cc-4422-807a-189a82015898 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:52.290671 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188bdc0-28cc-4422-807a-189a82015898 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:52.290948 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188bdc0-28cc-4422-807a-189a82015898 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:52.348208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:52.365697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5188bdc0-28cc-4422-807a-189a82015898 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:52.365697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5188bdc0-28cc-4422-807a-189a82015898 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:52.456598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 575/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:31:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:31:52.462514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af1ebb-6ed7-4fd6-b549-b32e9ade0594 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:52.560242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af1ebb-6ed7-4fd6-b549-b32e9ade0594 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:52.592810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a480f6-8d98-46ab-a25c-80c411973c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:52.593417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a480f6-8d98-46ab-a25c-80c411973c0e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:52.593727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a480f6-8d98-46ab-a25c-80c411973c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:52.594005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a480f6-8d98-46ab-a25c-80c411973c0e None admin] PATH_INFO: `/v3/projects/12cf9a3762c64ab2990640df89b01406` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:52.595120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a480f6-8d98-46ab-a25c-80c411973c0e None admin] RBAC: Authorizing `identity:get_project(project_id=12cf9a3762c64ab2990640df89b01406)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:52.596591 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a480f6-8d98-46ab-a25c-80c411973c0e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:52.598791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 585/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:31:52 2022] GET /identity/v3/projects/12cf9a3762c64ab2990640df89b01406 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:52.605005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188bdc0-28cc-4422-807a-189a82015898 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:52.694220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:52.703754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188bdc0-28cc-4422-807a-189a82015898 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:52.739151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5345e-74f9-478a-b1ad-e2516ab7fd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:52.739743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5345e-74f9-478a-b1ad-e2516ab7fd74 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:52.740025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5345e-74f9-478a-b1ad-e2516ab7fd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:52.740293 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5345e-74f9-478a-b1ad-e2516ab7fd74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:52.741014 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5345e-74f9-478a-b1ad-e2516ab7fd74 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=12cf9a3762c64ab2990640df89b01406, user.id=94427a9d17474af6a8f21162bb03709d) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:52.741324 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5345e-74f9-478a-b1ad-e2516ab7fd74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:52.742779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5345e-74f9-478a-b1ad-e2516ab7fd74 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:52.748625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 576/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:31:52 2022] GET /identity/v3/role_assignments?user.id=94427a9d17474af6a8f21162bb03709d&scope.project.id=12cf9a3762c64ab2990640df89b01406&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:54.771277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76463019-ab95-442e-96bd-f7a8f3c34a16 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:54.771842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76463019-ab95-442e-96bd-f7a8f3c34a16 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:54.771842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76463019-ab95-442e-96bd-f7a8f3c34a16 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:54.773141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 586/1162] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:54.778528 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711baf82-a4ba-449b-a2a0-a9ac9d0aae6d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:54.778985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711baf82-a4ba-449b-a2a0-a9ac9d0aae6d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:54.779087 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711baf82-a4ba-449b-a2a0-a9ac9d0aae6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:54.797429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-711baf82-a4ba-449b-a2a0-a9ac9d0aae6d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:54.798643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-711baf82-a4ba-449b-a2a0-a9ac9d0aae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:54.864150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 577/1163] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:31:54.885654 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76463019-ab95-442e-96bd-f7a8f3c34a16 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:54.920587 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:54.997572 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76463019-ab95-442e-96bd-f7a8f3c34a16 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:55.031169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605f639-9620-4c2e-96c6-dd8dd8a9f598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:55.031698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605f639-9620-4c2e-96c6-dd8dd8a9f598 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:55.031977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605f639-9620-4c2e-96c6-dd8dd8a9f598 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:55.032263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605f639-9620-4c2e-96c6-dd8dd8a9f598 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:55.099205 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605f639-9620-4c2e-96c6-dd8dd8a9f598 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:55.100656 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605f639-9620-4c2e-96c6-dd8dd8a9f598 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:55.132188 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:55.195233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 587/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:31:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:31:55.200957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711baf82-a4ba-449b-a2a0-a9ac9d0aae6d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:55.283248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:55.298985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711baf82-a4ba-449b-a2a0-a9ac9d0aae6d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:55.333832 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb1288-9aac-4bec-86b8-61077a7dac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:55.334471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb1288-9aac-4bec-86b8-61077a7dac13 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:55.334752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb1288-9aac-4bec-86b8-61077a7dac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:55.335023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb1288-9aac-4bec-86b8-61077a7dac13 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:55.341590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb1288-9aac-4bec-86b8-61077a7dac13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:55.343021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb1288-9aac-4bec-86b8-61077a7dac13 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-7aeb1288-9aac-4bec-86b8-61077a7dac13 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:31:55.354858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:31:55.356088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:31:55.356953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 578/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 9 18:31:55 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:31:55.362137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605f639-9620-4c2e-96c6-dd8dd8a9f598 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:55.460875 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605f639-9620-4c2e-96c6-dd8dd8a9f598 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:55.494639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca4c33-180e-4d31-80b2-5bf3fc3b5e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:55.495234 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca4c33-180e-4d31-80b2-5bf3fc3b5e9b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:55.495518 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca4c33-180e-4d31-80b2-5bf3fc3b5e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:55.495794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca4c33-180e-4d31-80b2-5bf3fc3b5e9b None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:55.496433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca4c33-180e-4d31-80b2-5bf3fc3b5e9b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:31:55.496743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca4c33-180e-4d31-80b2-5bf3fc3b5e9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:55.498164 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca4c33-180e-4d31-80b2-5bf3fc3b5e9b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:55.500542 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:55.505660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 588/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Tue Aug 9 18:31:55 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:31:57.520543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa298c-6202-41c5-a8dc-b41cdb2f0f16 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:57.521063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa298c-6202-41c5-a8dc-b41cdb2f0f16 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:57.521063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa298c-6202-41c5-a8dc-b41cdb2f0f16 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:57.522373 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 579/1167] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:57.528331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023554a6-88c4-4559-b5b8-2a058cfb656e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:57.528726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023554a6-88c4-4559-b5b8-2a058cfb656e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:57.528873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023554a6-88c4-4559-b5b8-2a058cfb656e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:57.547802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-023554a6-88c4-4559-b5b8-2a058cfb656e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:57.548961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-023554a6-88c4-4559-b5b8-2a058cfb656e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:57.614718 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 589/1168] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:31:57.636744 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa298c-6202-41c5-a8dc-b41cdb2f0f16 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:31:57.737617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa298c-6202-41c5-a8dc-b41cdb2f0f16 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:31:57.771201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63bc8f-d57c-465d-89bc-4a3588e72b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:31:57.771728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63bc8f-d57c-465d-89bc-4a3588e72b00 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:57.772008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63bc8f-d57c-465d-89bc-4a3588e72b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:57.772280 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63bc8f-d57c-465d-89bc-4a3588e72b00 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:57.772951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63bc8f-d57c-465d-89bc-4a3588e72b00 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:31:57.774328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63bc8f-d57c-465d-89bc-4a3588e72b00 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:31:57.863619 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 580/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Tue Aug 9 18:31:57 2022] POST /identity/v3/domains => generated 267 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:31:59.898175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cf398-2350-4768-8164-f03389304aea None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:59.898776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cf398-2350-4768-8164-f03389304aea None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:59.898776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cf398-2350-4768-8164-f03389304aea None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:59.900046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 590/1170] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:31:59.905232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad37bb9-afff-4ed4-98dc-4f9109ac7673 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:31:59.905701 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad37bb9-afff-4ed4-98dc-4f9109ac7673 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:31:59.905800 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad37bb9-afff-4ed4-98dc-4f9109ac7673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:31:59.913461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:31:59.923968 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4ad37bb9-afff-4ed4-98dc-4f9109ac7673 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:31:59.929273 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4ad37bb9-afff-4ed4-98dc-4f9109ac7673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:31:59.995296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 581/1171] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:31:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:32:00.017980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cf398-2350-4768-8164-f03389304aea None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:00.119692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cf398-2350-4768-8164-f03389304aea None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:00.147599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:00.153617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef87e9-52b1-4461-a3a0-7037a9e92fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:00.154200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef87e9-52b1-4461-a3a0-7037a9e92fed None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:00.154478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef87e9-52b1-4461-a3a0-7037a9e92fed None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:00.154746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef87e9-52b1-4461-a3a0-7037a9e92fed None admin] PATH_INFO: `/v3/domains/cfd85a17c7bd40108c135b5f4e3242f9` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:00.167129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef87e9-52b1-4461-a3a0-7037a9e92fed None admin] RBAC: Authorizing `identity:get_domain(domain_id=cfd85a17c7bd40108c135b5f4e3242f9)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:00.168785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef87e9-52b1-4461-a3a0-7037a9e92fed None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:00.170398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 591/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Tue Aug 9 18:32:00 2022] GET /identity/v3/domains/cfd85a17c7bd40108c135b5f4e3242f9 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:00.176849 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad37bb9-afff-4ed4-98dc-4f9109ac7673 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:00.272305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad37bb9-afff-4ed4-98dc-4f9109ac7673 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:00.307486 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389321f-cd50-44ad-8df1-e3f8d16ccfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:00.308095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389321f-cd50-44ad-8df1-e3f8d16ccfff None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:00.308375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389321f-cd50-44ad-8df1-e3f8d16ccfff None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:00.308648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389321f-cd50-44ad-8df1-e3f8d16ccfff None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:00.309304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389321f-cd50-44ad-8df1-e3f8d16ccfff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:00.310860 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389321f-cd50-44ad-8df1-e3f8d16ccfff None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:00.315633 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:00.416956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 582/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:32:00 2022] POST /identity/v3/projects => generated 361 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:32:02.465432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a779ee-4997-404f-8481-a74c609f8a78 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:02.466167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a779ee-4997-404f-8481-a74c609f8a78 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:02.466167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a779ee-4997-404f-8481-a74c609f8a78 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:02.467390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 592/1174] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:32:02.474039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0deee01-e456-4bdf-a8e3-ad40658da942 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:02.474039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0deee01-e456-4bdf-a8e3-ad40658da942 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:02.475160 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0deee01-e456-4bdf-a8e3-ad40658da942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:02.492566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f0deee01-e456-4bdf-a8e3-ad40658da942 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:32:02.493770 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f0deee01-e456-4bdf-a8e3-ad40658da942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:32:02.557646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 583/1175] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:32:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 18:32:02.582122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a779ee-4997-404f-8481-a74c609f8a78 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:02.684581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a779ee-4997-404f-8481-a74c609f8a78 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:02.717841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed55163-b9a9-4a86-8bb4-d3b89b470463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:02.718459 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed55163-b9a9-4a86-8bb4-d3b89b470463 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:02.718741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed55163-b9a9-4a86-8bb4-d3b89b470463 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:02.719013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed55163-b9a9-4a86-8bb4-d3b89b470463 None admin] PATH_INFO: `/v3/domains/cfd85a17c7bd40108c135b5f4e3242f9` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:02.720129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed55163-b9a9-4a86-8bb4-d3b89b470463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cfd85a17c7bd40108c135b5f4e3242f9)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:02.721580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed55163-b9a9-4a86-8bb4-d3b89b470463 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:02.723839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 593/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Tue Aug 9 18:32:02 2022] GET /identity/v3/domains/cfd85a17c7bd40108c135b5f4e3242f9 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:02.729639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0deee01-e456-4bdf-a8e3-ad40658da942 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:02.828619 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0deee01-e456-4bdf-a8e3-ad40658da942 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:02.845029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:02.861926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a46f22-8d09-4b2f-9eb5-c584a0acc4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:02.862519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a46f22-8d09-4b2f-9eb5-c584a0acc4f3 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:02.862796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a46f22-8d09-4b2f-9eb5-c584a0acc4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:02.863574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a46f22-8d09-4b2f-9eb5-c584a0acc4f3 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:02.864220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a46f22-8d09-4b2f-9eb5-c584a0acc4f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:02.865665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a46f22-8d09-4b2f-9eb5-c584a0acc4f3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:02.975523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 584/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:32:02 2022] POST /identity/v3/users => generated 311 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:32:05.067299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b73b60-a88a-45de-a6e1-f5fedde16eb8 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:05.067877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b73b60-a88a-45de-a6e1-f5fedde16eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:05.068269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b73b60-a88a-45de-a6e1-f5fedde16eb8 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:05.070753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 594/1178] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:32:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:32:05.076608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa1ba2-dee8-4cad-b232-4884eb76ff89 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:05.079634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa1ba2-dee8-4cad-b232-4884eb76ff89 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:05.079634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa1ba2-dee8-4cad-b232-4884eb76ff89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:05.095801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-dcaa1ba2-dee8-4cad-b232-4884eb76ff89 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:32:05.096986 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dcaa1ba2-dee8-4cad-b232-4884eb76ff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:32:05.182668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 585/1179] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:32:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:32:05.206472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b73b60-a88a-45de-a6e1-f5fedde16eb8 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:05.270910 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:05.326009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b73b60-a88a-45de-a6e1-f5fedde16eb8 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:05.364079 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f1a0d-c3e2-40b9-b2de-1c0dc4800226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:05.364704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f1a0d-c3e2-40b9-b2de-1c0dc4800226 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:05.364983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f1a0d-c3e2-40b9-b2de-1c0dc4800226 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:05.365252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f1a0d-c3e2-40b9-b2de-1c0dc4800226 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:05.376037 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f1a0d-c3e2-40b9-b2de-1c0dc4800226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-5b9f1a0d-c3e2-40b9-b2de-1c0dc4800226 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:32:05.380869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:32:05.382112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:32:05.383201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 595/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:32:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:32:05.390088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa1ba2-dee8-4cad-b232-4884eb76ff89 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:05.499499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa1ba2-dee8-4cad-b232-4884eb76ff89 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:05.501048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:05.541836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc379f-56ac-46cd-a66f-4b7450172bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:05.542457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc379f-56ac-46cd-a66f-4b7450172bda None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:05.542738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc379f-56ac-46cd-a66f-4b7450172bda None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:05.543176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc379f-56ac-46cd-a66f-4b7450172bda None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:05.543856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc379f-56ac-46cd-a66f-4b7450172bda None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:32:05.544165 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc379f-56ac-46cd-a66f-4b7450172bda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:05.545565 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc379f-56ac-46cd-a66f-4b7450172bda None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:05.554160 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 586/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:32:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:05.560154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f1a0d-c3e2-40b9-b2de-1c0dc4800226 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:05.672898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f1a0d-c3e2-40b9-b2de-1c0dc4800226 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:05.692626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:05.711750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc71fcc-37f1-4987-b237-f52b12f4e61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:05.712693 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc71fcc-37f1-4987-b237-f52b12f4e61d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:05.712987 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc71fcc-37f1-4987-b237-f52b12f4e61d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:05.713266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc71fcc-37f1-4987-b237-f52b12f4e61d None admin] PATH_INFO: `/v3/users/33e61109771d4f809722a79705eed15f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:05.728377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc71fcc-37f1-4987-b237-f52b12f4e61d None admin] RBAC: Authorizing `identity:get_user(user_id=33e61109771d4f809722a79705eed15f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:05.729877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc71fcc-37f1-4987-b237-f52b12f4e61d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:05.731537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 596/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:32:05 2022] GET /identity/v3/users/33e61109771d4f809722a79705eed15f => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:05.739698 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc379f-56ac-46cd-a66f-4b7450172bda None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:05.855969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc379f-56ac-46cd-a66f-4b7450172bda None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:05.889001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896a62b-79d2-4bbd-b05c-80752143d24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:05.889636 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896a62b-79d2-4bbd-b05c-80752143d24f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:05.889927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896a62b-79d2-4bbd-b05c-80752143d24f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:05.890203 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896a62b-79d2-4bbd-b05c-80752143d24f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:05.921061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:05.959716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896a62b-79d2-4bbd-b05c-80752143d24f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:05.961073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896a62b-79d2-4bbd-b05c-80752143d24f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:06.056995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 587/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:32:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:32:06.063446 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc71fcc-37f1-4987-b237-f52b12f4e61d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:06.163357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc71fcc-37f1-4987-b237-f52b12f4e61d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:06.196644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5493c58-c601-4195-833e-504ac4fc055c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:06.197269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5493c58-c601-4195-833e-504ac4fc055c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:06.197568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5493c58-c601-4195-833e-504ac4fc055c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:06.197844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5493c58-c601-4195-833e-504ac4fc055c None admin] PATH_INFO: `/v3/projects/1d205fdad3a74a90ad003fe0d1f7fae3` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:06.198953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5493c58-c601-4195-833e-504ac4fc055c None admin] RBAC: Authorizing `identity:get_project(project_id=1d205fdad3a74a90ad003fe0d1f7fae3)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:06.200387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5493c58-c601-4195-833e-504ac4fc055c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:06.202060 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 597/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:32:06 2022] GET /identity/v3/projects/1d205fdad3a74a90ad003fe0d1f7fae3 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:06.208848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896a62b-79d2-4bbd-b05c-80752143d24f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:06.273875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:06.311777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896a62b-79d2-4bbd-b05c-80752143d24f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:06.346965 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a603a2-5c8d-4c10-b772-90f035859c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:06.347558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a603a2-5c8d-4c10-b772-90f035859c65 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:06.347840 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a603a2-5c8d-4c10-b772-90f035859c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:06.348118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a603a2-5c8d-4c10-b772-90f035859c65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:06.348837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a603a2-5c8d-4c10-b772-90f035859c65 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=1d205fdad3a74a90ad003fe0d1f7fae3, user.id=33e61109771d4f809722a79705eed15f) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:32:06.349144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a603a2-5c8d-4c10-b772-90f035859c65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:06.350734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a603a2-5c8d-4c10-b772-90f035859c65 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:06.355908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 588/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:32:06 2022] GET /identity/v3/role_assignments?user.id=33e61109771d4f809722a79705eed15f&scope.project.id=1d205fdad3a74a90ad003fe0d1f7fae3&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:08.352624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676a6ed-8846-4408-be6a-e5575ef8a3ec None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:08.353190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676a6ed-8846-4408-be6a-e5575ef8a3ec None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:08.353190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676a6ed-8846-4408-be6a-e5575ef8a3ec None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:08.354553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 598/1186] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:32:08.360377 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e22d5a-6147-410c-9705-19f1d6c83651 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:08.360799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e22d5a-6147-410c-9705-19f1d6c83651 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:08.360906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e22d5a-6147-410c-9705-19f1d6c83651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:08.379912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-47e22d5a-6147-410c-9705-19f1d6c83651 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:32:08.381085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-47e22d5a-6147-410c-9705-19f1d6c83651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:32:08.446363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 589/1187] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:32:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:32:08.469496 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676a6ed-8846-4408-be6a-e5575ef8a3ec None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:08.516080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:08.577033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676a6ed-8846-4408-be6a-e5575ef8a3ec None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:08.610141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f7307-934c-4917-b1c2-e71495ff9a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:08.610745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f7307-934c-4917-b1c2-e71495ff9a00 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:08.611025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f7307-934c-4917-b1c2-e71495ff9a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:08.611299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f7307-934c-4917-b1c2-e71495ff9a00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:08.621614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f7307-934c-4917-b1c2-e71495ff9a00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-e34f7307-934c-4917-b1c2-e71495ff9a00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:32:08.626270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:32:08.627360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:32:08.629013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 599/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:32:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:32:08.635110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e22d5a-6147-410c-9705-19f1d6c83651 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:08.692786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:08.733062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e22d5a-6147-410c-9705-19f1d6c83651 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:08.766405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91888b66-a064-4b76-895c-2dea08385ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:08.767003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91888b66-a064-4b76-895c-2dea08385ece None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:08.767274 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91888b66-a064-4b76-895c-2dea08385ece None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:08.767543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91888b66-a064-4b76-895c-2dea08385ece None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:08.768198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91888b66-a064-4b76-895c-2dea08385ece None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:32:08.768492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91888b66-a064-4b76-895c-2dea08385ece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:08.769941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91888b66-a064-4b76-895c-2dea08385ece None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:08.778252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 590/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:32:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:08.784174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f7307-934c-4917-b1c2-e71495ff9a00 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:08.882913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f7307-934c-4917-b1c2-e71495ff9a00 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:08.885091 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:08.916964 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac1bf7-0840-4941-a114-7a51275ae8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:08.917796 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac1bf7-0840-4941-a114-7a51275ae8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:08.918134 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac1bf7-0840-4941-a114-7a51275ae8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:08.918598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac1bf7-0840-4941-a114-7a51275ae8c9 None admin] PATH_INFO: `/v3/users/33e61109771d4f809722a79705eed15f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:08.919980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac1bf7-0840-4941-a114-7a51275ae8c9 None admin] RBAC: Authorizing `identity:get_user(user_id=33e61109771d4f809722a79705eed15f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:08.921412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac1bf7-0840-4941-a114-7a51275ae8c9 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:08.923091 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 600/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:32:08 2022] GET /identity/v3/users/33e61109771d4f809722a79705eed15f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:08.929960 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91888b66-a064-4b76-895c-2dea08385ece None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:09.027755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91888b66-a064-4b76-895c-2dea08385ece None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:09.048141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:09.062988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caca23-54fe-455b-86fa-77c4cdf66f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:09.063617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caca23-54fe-455b-86fa-77c4cdf66f0a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:09.063900 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caca23-54fe-455b-86fa-77c4cdf66f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:09.064173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caca23-54fe-455b-86fa-77c4cdf66f0a None admin] PATH_INFO: `/v3/projects/1d205fdad3a74a90ad003fe0d1f7fae3` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:09.065284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98caca23-54fe-455b-86fa-77c4cdf66f0a None admin] RBAC: Authorizing `identity:get_project(project_id=1d205fdad3a74a90ad003fe0d1f7fae3)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:09.066771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98caca23-54fe-455b-86fa-77c4cdf66f0a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:09.068390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 591/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:32:08 2022] GET /identity/v3/projects/1d205fdad3a74a90ad003fe0d1f7fae3 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:09.074479 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac1bf7-0840-4941-a114-7a51275ae8c9 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:09.184713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac1bf7-0840-4941-a114-7a51275ae8c9 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:09.217603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88b837-21ac-4d49-9c8f-130171199945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:09.218252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88b837-21ac-4d49-9c8f-130171199945 None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:09.218536 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88b837-21ac-4d49-9c8f-130171199945 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:09.218814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88b837-21ac-4d49-9c8f-130171199945 None admin] PATH_INFO: `/v3/projects/1d205fdad3a74a90ad003fe0d1f7fae3/users/33e61109771d4f809722a79705eed15f/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:09.220723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88b837-21ac-4d49-9c8f-130171199945 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d205fdad3a74a90ad003fe0d1f7fae3, user_id=33e61109771d4f809722a79705eed15f, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:09.222716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88b837-21ac-4d49-9c8f-130171199945 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:09.273456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 601/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:32:09 2022] PUT /identity/v3/projects/1d205fdad3a74a90ad003fe0d1f7fae3/users/33e61109771d4f809722a79705eed15f/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:32:11.285214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe1866-8c61-4d6a-ae9b-96aad8822792 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:11.285741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe1866-8c61-4d6a-ae9b-96aad8822792 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:11.285844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe1866-8c61-4d6a-ae9b-96aad8822792 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:11.287189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 592/1193] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:32:11.292264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffec6e-f0a1-4f08-bb03-9261c8b0167e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:11.292709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffec6e-f0a1-4f08-bb03-9261c8b0167e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:11.292818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffec6e-f0a1-4f08-bb03-9261c8b0167e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:11.311275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b5ffec6e-f0a1-4f08-bb03-9261c8b0167e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:32:11.312452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b5ffec6e-f0a1-4f08-bb03-9261c8b0167e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:32:11.337345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:11.379101 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 602/1194] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:32:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:32:11.401848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe1866-8c61-4d6a-ae9b-96aad8822792 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:11.504242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe1866-8c61-4d6a-ae9b-96aad8822792 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:11.538178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fd93d-4d68-4b8b-89e8-f886ba1c5d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:11.538735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fd93d-4d68-4b8b-89e8-f886ba1c5d84 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:11.539015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fd93d-4d68-4b8b-89e8-f886ba1c5d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:11.539287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fd93d-4d68-4b8b-89e8-f886ba1c5d84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:11.550631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7fd93d-4d68-4b8b-89e8-f886ba1c5d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-0e7fd93d-4d68-4b8b-89e8-f886ba1c5d84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:32:11.559808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:32:11.560909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:32:11.561804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 593/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:32:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:32:11.567194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffec6e-f0a1-4f08-bb03-9261c8b0167e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:11.665081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffec6e-f0a1-4f08-bb03-9261c8b0167e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:11.698253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7cd3db-abc6-4e07-b7ad-622a39ef806e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:11.698851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7cd3db-abc6-4e07-b7ad-622a39ef806e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:11.699131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7cd3db-abc6-4e07-b7ad-622a39ef806e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:11.699406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7cd3db-abc6-4e07-b7ad-622a39ef806e None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:11.700067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7cd3db-abc6-4e07-b7ad-622a39ef806e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:32:11.700372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7cd3db-abc6-4e07-b7ad-622a39ef806e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:11.701753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7cd3db-abc6-4e07-b7ad-622a39ef806e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:11.710658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 603/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:32:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:11.716613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fd93d-4d68-4b8b-89e8-f886ba1c5d84 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:11.765441 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:11.816353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fd93d-4d68-4b8b-89e8-f886ba1c5d84 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:11.849578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfa465-f6fa-42a3-9c66-d1243d29ad8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:11.850200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfa465-f6fa-42a3-9c66-d1243d29ad8a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:11.850482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfa465-f6fa-42a3-9c66-d1243d29ad8a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:11.850756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfa465-f6fa-42a3-9c66-d1243d29ad8a None admin] PATH_INFO: `/v3/users/33e61109771d4f809722a79705eed15f` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:11.851865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfa465-f6fa-42a3-9c66-d1243d29ad8a None admin] RBAC: Authorizing `identity:get_user(user_id=33e61109771d4f809722a79705eed15f)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:11.853285 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfa465-f6fa-42a3-9c66-d1243d29ad8a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:11.854861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 594/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Tue Aug 9 18:32:11 2022] GET /identity/v3/users/33e61109771d4f809722a79705eed15f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:11.860657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7cd3db-abc6-4e07-b7ad-622a39ef806e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:11.885479 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:11.965109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7cd3db-abc6-4e07-b7ad-622a39ef806e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:11.998077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8a31e-5ad2-4942-9bfc-81054c7d2cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:11.998667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8a31e-5ad2-4942-9bfc-81054c7d2cdf None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:11.998942 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8a31e-5ad2-4942-9bfc-81054c7d2cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:11.999212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8a31e-5ad2-4942-9bfc-81054c7d2cdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:12.064689 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8a31e-5ad2-4942-9bfc-81054c7d2cdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:12.066741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8a31e-5ad2-4942-9bfc-81054c7d2cdf None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:12.090248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:12.161938 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 604/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:32:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:32:12.167953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfa465-f6fa-42a3-9c66-d1243d29ad8a None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:12.266122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfa465-f6fa-42a3-9c66-d1243d29ad8a None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:12.283449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:12.301552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceba85a-2ed9-497e-a80c-733decfb2f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:12.302176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceba85a-2ed9-497e-a80c-733decfb2f7d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:12.302456 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceba85a-2ed9-497e-a80c-733decfb2f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:12.302727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceba85a-2ed9-497e-a80c-733decfb2f7d None admin] PATH_INFO: `/v3/projects/1d205fdad3a74a90ad003fe0d1f7fae3` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:12.304086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceba85a-2ed9-497e-a80c-733decfb2f7d None admin] RBAC: Authorizing `identity:get_project(project_id=1d205fdad3a74a90ad003fe0d1f7fae3)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:12.305558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceba85a-2ed9-497e-a80c-733decfb2f7d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:12.307887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 595/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Tue Aug 9 18:32:12 2022] GET /identity/v3/projects/1d205fdad3a74a90ad003fe0d1f7fae3 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:12.313611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8a31e-5ad2-4942-9bfc-81054c7d2cdf None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:32:12.410942 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8a31e-5ad2-4942-9bfc-81054c7d2cdf None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:32:12.433752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:12.444720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107fbd8-8747-43b4-b496-fc94d449857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:32:12.451603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107fbd8-8747-43b4-b496-fc94d449857b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:32:12.451603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107fbd8-8747-43b4-b496-fc94d449857b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:32:12.451603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107fbd8-8747-43b4-b496-fc94d449857b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:32:12.451603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107fbd8-8747-43b4-b496-fc94d449857b None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=1d205fdad3a74a90ad003fe0d1f7fae3, user.id=33e61109771d4f809722a79705eed15f) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:32:12.451603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107fbd8-8747-43b4-b496-fc94d449857b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:32:12.451603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107fbd8-8747-43b4-b496-fc94d449857b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:32:12.454000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 605/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:32:12 2022] GET /identity/v3/role_assignments?user.id=33e61109771d4f809722a79705eed15f&scope.project.id=1d205fdad3a74a90ad003fe0d1f7fae3&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:32:22.308377 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=8 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:32:22.461095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:26.445090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644e57d-91d5-45bd-b651-27464da0f8aa None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:26.445733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644e57d-91d5-45bd-b651-27464da0f8aa None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:26.445733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644e57d-91d5-45bd-b651-27464da0f8aa None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:26.447026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 596/1201] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:33:26.452233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b99e02-f1ab-4c83-8bef-1c914b6bd931 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:26.452795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b99e02-f1ab-4c83-8bef-1c914b6bd931 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:26.452795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b99e02-f1ab-4c83-8bef-1c914b6bd931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:26.477461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f5b99e02-f1ab-4c83-8bef-1c914b6bd931 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:26.478725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f5b99e02-f1ab-4c83-8bef-1c914b6bd931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:26.547126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 606/1202] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:33:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:33:26.571439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644e57d-91d5-45bd-b651-27464da0f8aa None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:26.670356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644e57d-91d5-45bd-b651-27464da0f8aa None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:26.703238 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841e013-963f-4e68-9d29-2be03b26ac1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:26.703884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841e013-963f-4e68-9d29-2be03b26ac1e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:26.704166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841e013-963f-4e68-9d29-2be03b26ac1e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:26.704439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841e013-963f-4e68-9d29-2be03b26ac1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:26.715161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841e013-963f-4e68-9d29-2be03b26ac1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:26.716629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841e013-963f-4e68-9d29-2be03b26ac1e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-5841e013-963f-4e68-9d29-2be03b26ac1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:26.731305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:26.732461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:26.733896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:26.733896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:26.733896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:26.733896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 597/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:26.738517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b99e02-f1ab-4c83-8bef-1c914b6bd931 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:26.837260 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b99e02-f1ab-4c83-8bef-1c914b6bd931 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:26.865375 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:26.870965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f3447-54f3-41bd-9793-9b4d141bc55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:26.871560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f3447-54f3-41bd-9793-9b4d141bc55c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:26.871834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f3447-54f3-41bd-9793-9b4d141bc55c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:26.872099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f3447-54f3-41bd-9793-9b4d141bc55c None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:26.872734 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f3447-54f3-41bd-9793-9b4d141bc55c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:26.873035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f3447-54f3-41bd-9793-9b4d141bc55c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:26.874500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f3447-54f3-41bd-9793-9b4d141bc55c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:26.886004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 607/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:26.891928 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841e013-963f-4e68-9d29-2be03b26ac1e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:26.944588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:26.991355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841e013-963f-4e68-9d29-2be03b26ac1e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:27.025445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09b230-7064-4168-9c74-dca89ddbffb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:27.026066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09b230-7064-4168-9c74-dca89ddbffb3 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:27.026342 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09b230-7064-4168-9c74-dca89ddbffb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:27.026612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09b230-7064-4168-9c74-dca89ddbffb3 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:27.027258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09b230-7064-4168-9c74-dca89ddbffb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:27.028645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09b230-7064-4168-9c74-dca89ddbffb3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:27.111370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 598/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Tue Aug 9 18:33:26 2022] POST /identity/v3/users => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:33:29.163688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd76c6-7eee-4139-946e-279d70442794 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:29.164282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd76c6-7eee-4139-946e-279d70442794 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:29.164282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd76c6-7eee-4139-946e-279d70442794 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:29.165674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 608/1206] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:33:29.170917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803ff69-5946-4ce8-aeb1-70b1ca53d674 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:29.171318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803ff69-5946-4ce8-aeb1-70b1ca53d674 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:29.171466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803ff69-5946-4ce8-aeb1-70b1ca53d674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:29.190375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3803ff69-5946-4ce8-aeb1-70b1ca53d674 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:29.191554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3803ff69-5946-4ce8-aeb1-70b1ca53d674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:29.257980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 599/1207] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:33:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:33:29.281604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd76c6-7eee-4139-946e-279d70442794 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:29.392965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd76c6-7eee-4139-946e-279d70442794 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:29.426825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9fed4-280b-4a41-b4d1-25a06bce597d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:29.427627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9fed4-280b-4a41-b4d1-25a06bce597d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:29.427900 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9fed4-280b-4a41-b4d1-25a06bce597d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:29.428238 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9fed4-280b-4a41-b4d1-25a06bce597d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:29.438340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9fed4-280b-4a41-b4d1-25a06bce597d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-cca9fed4-280b-4a41-b4d1-25a06bce597d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:33:29.443126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:29.450523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:29.451565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 609/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:33:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:29.452434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803ff69-5946-4ce8-aeb1-70b1ca53d674 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:29.490841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:29.549905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803ff69-5946-4ce8-aeb1-70b1ca53d674 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:29.583787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cbc7d-9a2e-4750-ae9d-20111d8bce34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:29.584399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cbc7d-9a2e-4750-ae9d-20111d8bce34 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:29.584893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cbc7d-9a2e-4750-ae9d-20111d8bce34 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:29.585764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cbc7d-9a2e-4750-ae9d-20111d8bce34 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:29.586443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492cbc7d-9a2e-4750-ae9d-20111d8bce34 None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:29.586752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492cbc7d-9a2e-4750-ae9d-20111d8bce34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:29.588126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492cbc7d-9a2e-4750-ae9d-20111d8bce34 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:29.596040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 600/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:33:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:29.603253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9fed4-280b-4a41-b4d1-25a06bce597d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:29.652820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:29.701862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9fed4-280b-4a41-b4d1-25a06bce597d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:29.735726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff8085-b717-409d-aa5d-5c89328f84ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:29.736350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff8085-b717-409d-aa5d-5c89328f84ba None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:29.736631 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff8085-b717-409d-aa5d-5c89328f84ba None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:29.736904 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff8085-b717-409d-aa5d-5c89328f84ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:29.747355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff8085-b717-409d-aa5d-5c89328f84ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:29.748799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff8085-b717-409d-aa5d-5c89328f84ba None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-77ff8085-b717-409d-aa5d-5c89328f84ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:29.762416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:29.763573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:29.764794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:29.764794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:29.764794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:29.764794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 610/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:33:29.769232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cbc7d-9a2e-4750-ae9d-20111d8bce34 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:29.865772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:29.868629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cbc7d-9a2e-4750-ae9d-20111d8bce34 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:29.901995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0705b4-69a5-41c9-9ba2-3b9ededfcd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:29.902682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0705b4-69a5-41c9-9ba2-3b9ededfcd6c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:29.902963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0705b4-69a5-41c9-9ba2-3b9ededfcd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:29.903236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0705b4-69a5-41c9-9ba2-3b9ededfcd6c None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:29.903906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0705b4-69a5-41c9-9ba2-3b9ededfcd6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:29.904211 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0705b4-69a5-41c9-9ba2-3b9ededfcd6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:29.905588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0705b4-69a5-41c9-9ba2-3b9ededfcd6c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:29.916943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 601/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:29.922583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff8085-b717-409d-aa5d-5c89328f84ba None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:30.026542 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff8085-b717-409d-aa5d-5c89328f84ba None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:30.028007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:30.072707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c05228-63db-4af0-94d8-e77c4d7d43c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:30.073337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c05228-63db-4af0-94d8-e77c4d7d43c3 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:30.073640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c05228-63db-4af0-94d8-e77c4d7d43c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:30.073911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c05228-63db-4af0-94d8-e77c4d7d43c3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:30.080761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c05228-63db-4af0-94d8-e77c4d7d43c3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:30.082241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c05228-63db-4af0-94d8-e77c4d7d43c3 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b5c05228-63db-4af0-94d8-e77c4d7d43c3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:33:30.094068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:30.095209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:30.096333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:30.096333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:30.096333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:33:30.096333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:33:30.096333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:33:30.096333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:33:30.096333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:33:30.096333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:33:30.096333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:33:30.096333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:30.096333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 611/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:33:30.101388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0705b4-69a5-41c9-9ba2-3b9ededfcd6c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:30.206891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0705b4-69a5-41c9-9ba2-3b9ededfcd6c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:30.242336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee296f7-cacd-4623-9259-15d197c72eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:30.242954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee296f7-cacd-4623-9259-15d197c72eed None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:30.243234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee296f7-cacd-4623-9259-15d197c72eed None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:30.243505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee296f7-cacd-4623-9259-15d197c72eed None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:30.244242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee296f7-cacd-4623-9259-15d197c72eed None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:30.244550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee296f7-cacd-4623-9259-15d197c72eed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:30.246013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee296f7-cacd-4623-9259-15d197c72eed None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:30.269295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 602/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Aug 9 18:33:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:30.270697 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c05228-63db-4af0-94d8-e77c4d7d43c3 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:30.377069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c05228-63db-4af0-94d8-e77c4d7d43c3 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:30.417546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f34b7-3e4c-44b9-b154-9924df51bf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:30.418175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f34b7-3e4c-44b9-b154-9924df51bf36 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:30.418453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f34b7-3e4c-44b9-b154-9924df51bf36 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:30.418722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f34b7-3e4c-44b9-b154-9924df51bf36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:30.453109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:30.495927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f34b7-3e4c-44b9-b154-9924df51bf36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:30.497441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f34b7-3e4c-44b9-b154-9924df51bf36 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:30.607098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 612/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:33:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:33:30.613176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee296f7-cacd-4623-9259-15d197c72eed None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:30.632788 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:30.716935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee296f7-cacd-4623-9259-15d197c72eed None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:30.751551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b82e22-ac07-4894-ad04-5a6c19294741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:30.752163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b82e22-ac07-4894-ad04-5a6c19294741 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:30.752440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b82e22-ac07-4894-ad04-5a6c19294741 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:30.752706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b82e22-ac07-4894-ad04-5a6c19294741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:30.763420 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b82e22-ac07-4894-ad04-5a6c19294741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:30.770831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b82e22-ac07-4894-ad04-5a6c19294741 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-e1b82e22-ac07-4894-ad04-5a6c19294741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:30.779076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:30.785003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:30.786644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:30.786644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:30.786644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:30.786644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 603/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:30.787013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f34b7-3e4c-44b9-b154-9924df51bf36 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:30.851248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:30.891528 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f34b7-3e4c-44b9-b154-9924df51bf36 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:30.926387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc86b6-f690-40a5-9121-3be4dec55ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:30.926995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc86b6-f690-40a5-9121-3be4dec55ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:30.927543 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc86b6-f690-40a5-9121-3be4dec55ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:30.927816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc86b6-f690-40a5-9121-3be4dec55ef1 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:30.928456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc86b6-f690-40a5-9121-3be4dec55ef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:30.928758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc86b6-f690-40a5-9121-3be4dec55ef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:30.930393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc86b6-f690-40a5-9121-3be4dec55ef1 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:30.942675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 613/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:30.948304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b82e22-ac07-4894-ad04-5a6c19294741 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:31.017562 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:31.046790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b82e22-ac07-4894-ad04-5a6c19294741 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:31.079947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1556e4-f468-4fef-8839-dfe466e24d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:31.080569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1556e4-f468-4fef-8839-dfe466e24d33 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:31.080841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1556e4-f468-4fef-8839-dfe466e24d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:31.081108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1556e4-f468-4fef-8839-dfe466e24d33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:31.087905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1556e4-f468-4fef-8839-dfe466e24d33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:31.089363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1556e4-f468-4fef-8839-dfe466e24d33 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-bb1556e4-f468-4fef-8839-dfe466e24d33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:33:31.100778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:31.102537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:31.104047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:33:31.104047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:33:31.104047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:33:31.104047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:33:31.104047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:33:31.104047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:33:31.104047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:33:31.104047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:31.104047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 604/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:33:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:31.107809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc86b6-f690-40a5-9121-3be4dec55ef1 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:31.206728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc86b6-f690-40a5-9121-3be4dec55ef1 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:31.230130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:31.242114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca177d-458b-4ff7-80d6-592ab9e68ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:31.242726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca177d-458b-4ff7-80d6-592ab9e68ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:31.243006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca177d-458b-4ff7-80d6-592ab9e68ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:31.243274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca177d-458b-4ff7-80d6-592ab9e68ac7 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:31.243965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca177d-458b-4ff7-80d6-592ab9e68ac7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:31.244268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca177d-458b-4ff7-80d6-592ab9e68ac7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:31.245756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca177d-458b-4ff7-80d6-592ab9e68ac7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:31.258217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 614/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:33:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:31.264532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1556e4-f468-4fef-8839-dfe466e24d33 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:31.366923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1556e4-f468-4fef-8839-dfe466e24d33 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:31.394807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:31.400459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7aaf9-4a86-4853-bcd7-c72b3de43d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:31.401062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7aaf9-4a86-4853-bcd7-c72b3de43d4f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:31.401339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7aaf9-4a86-4853-bcd7-c72b3de43d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:31.401645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7aaf9-4a86-4853-bcd7-c72b3de43d4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:31.402373 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7aaf9-4a86-4853-bcd7-c72b3de43d4f None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=5e0aa6b215f646c280af245a89ccf849) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:31.402699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7aaf9-4a86-4853-bcd7-c72b3de43d4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:31.404117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7aaf9-4a86-4853-bcd7-c72b3de43d4f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:31.409752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 605/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:33:31 2022] GET /identity/v3/role_assignments?user.id=5e0aa6b215f646c280af245a89ccf849&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:33.426166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22186ce5-a4c3-404f-b5bf-1ea3a01988bc None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:33.426776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22186ce5-a4c3-404f-b5bf-1ea3a01988bc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:33.426776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22186ce5-a4c3-404f-b5bf-1ea3a01988bc None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:33.428598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 615/1220] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:33:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:33:33.433787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891cadb-110f-4a5f-8a8b-f9b4c8ae1773 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:33.437861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891cadb-110f-4a5f-8a8b-f9b4c8ae1773 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:33.437861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891cadb-110f-4a5f-8a8b-f9b4c8ae1773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:33.453000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e891cadb-110f-4a5f-8a8b-f9b4c8ae1773 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:33.454303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e891cadb-110f-4a5f-8a8b-f9b4c8ae1773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:33.519892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 606/1221] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:33:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:33:33.542751 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22186ce5-a4c3-404f-b5bf-1ea3a01988bc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:33.644760 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22186ce5-a4c3-404f-b5bf-1ea3a01988bc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:33.679799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba93309-45c3-4bd3-ba1d-c19317e54710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:33.680427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba93309-45c3-4bd3-ba1d-c19317e54710 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:33.680707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba93309-45c3-4bd3-ba1d-c19317e54710 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:33.680976 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba93309-45c3-4bd3-ba1d-c19317e54710 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:33.691120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba93309-45c3-4bd3-ba1d-c19317e54710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-eba93309-45c3-4bd3-ba1d-c19317e54710 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:33:33.695907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:33.697022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:33.698322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 616/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:33:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:33.703145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891cadb-110f-4a5f-8a8b-f9b4c8ae1773 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:33.799234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891cadb-110f-4a5f-8a8b-f9b4c8ae1773 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:33.822554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:33.833159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a7428-df5f-4f44-9643-17176b6770e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:33.833796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a7428-df5f-4f44-9643-17176b6770e7 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:33.834077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a7428-df5f-4f44-9643-17176b6770e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:33.834346 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a7428-df5f-4f44-9643-17176b6770e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:33.835010 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a7428-df5f-4f44-9643-17176b6770e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:33.835314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a7428-df5f-4f44-9643-17176b6770e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:33.836708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a7428-df5f-4f44-9643-17176b6770e7 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:33.844928 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 607/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:33:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:33.851321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba93309-45c3-4bd3-ba1d-c19317e54710 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:33.895587 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:33.953973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba93309-45c3-4bd3-ba1d-c19317e54710 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:33.989019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e68bba-e13f-4337-9bd6-223ae6aa6afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:33.989712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68bba-e13f-4337-9bd6-223ae6aa6afe None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:33.989985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68bba-e13f-4337-9bd6-223ae6aa6afe None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:33.990249 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68bba-e13f-4337-9bd6-223ae6aa6afe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:34.000820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e68bba-e13f-4337-9bd6-223ae6aa6afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:34.003670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e68bba-e13f-4337-9bd6-223ae6aa6afe None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-16e68bba-e13f-4337-9bd6-223ae6aa6afe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:34.016510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:34.017794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:34.018907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:34.018907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:34.018907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:34.018907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 617/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:34.024083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a7428-df5f-4f44-9643-17176b6770e7 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:34.129940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a7428-df5f-4f44-9643-17176b6770e7 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:34.164656 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899ab47-b129-4734-8da1-c50fd557089d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:34.165269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899ab47-b129-4734-8da1-c50fd557089d None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:34.165638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899ab47-b129-4734-8da1-c50fd557089d None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:34.165920 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899ab47-b129-4734-8da1-c50fd557089d None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:34.166568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899ab47-b129-4734-8da1-c50fd557089d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:34.166873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899ab47-b129-4734-8da1-c50fd557089d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:34.168230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899ab47-b129-4734-8da1-c50fd557089d None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:34.180521 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 608/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:34.186388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e68bba-e13f-4337-9bd6-223ae6aa6afe None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:34.280393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:34.287050 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e68bba-e13f-4337-9bd6-223ae6aa6afe None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:34.320053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9827f4-e133-42a2-b20f-41a70681ac99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:34.320870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9827f4-e133-42a2-b20f-41a70681ac99 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:34.321015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9827f4-e133-42a2-b20f-41a70681ac99 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:34.321214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9827f4-e133-42a2-b20f-41a70681ac99 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:34.328016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9827f4-e133-42a2-b20f-41a70681ac99 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:34.329477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9827f4-e133-42a2-b20f-41a70681ac99 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-db9827f4-e133-42a2-b20f-41a70681ac99 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:33:34.341202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:34.342370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:34.343385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:34.343385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:34.343385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:33:34.343385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:33:34.343385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:33:34.343385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:33:34.343385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:33:34.343385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:33:34.343385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:33:34.343385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:34.343385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 618/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:34.348346 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899ab47-b129-4734-8da1-c50fd557089d None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:34.401435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:34.454225 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899ab47-b129-4734-8da1-c50fd557089d None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:34.487507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130d423-96a2-4062-9c27-a876c3245891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:34.488112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d423-96a2-4062-9c27-a876c3245891 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:34.488391 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d423-96a2-4062-9c27-a876c3245891 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:34.488666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d423-96a2-4062-9c27-a876c3245891 None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:34.489419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130d423-96a2-4062-9c27-a876c3245891 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:34.489757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130d423-96a2-4062-9c27-a876c3245891 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:34.491222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130d423-96a2-4062-9c27-a876c3245891 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:34.509314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 609/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Aug 9 18:33:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:34.515239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9827f4-e133-42a2-b20f-41a70681ac99 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:34.615272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9827f4-e133-42a2-b20f-41a70681ac99 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:34.648333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4556e-a52f-4839-8b3e-a6bf9ac3d05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:34.648950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4556e-a52f-4839-8b3e-a6bf9ac3d05a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:34.649230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4556e-a52f-4839-8b3e-a6bf9ac3d05a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:34.649502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4556e-a52f-4839-8b3e-a6bf9ac3d05a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:34.659854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4556e-a52f-4839-8b3e-a6bf9ac3d05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:34.661263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4556e-a52f-4839-8b3e-a6bf9ac3d05a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-4fe4556e-a52f-4839-8b3e-a6bf9ac3d05a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:34.676713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:34.678416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:34.679299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:34.679299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:34.679299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:34.679299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 619/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:34.684680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130d423-96a2-4062-9c27-a876c3245891 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:34.785964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130d423-96a2-4062-9c27-a876c3245891 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:34.788245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:34.819950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05386aa6-6acb-4557-a866-d84e7104dc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:34.820611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05386aa6-6acb-4557-a866-d84e7104dc8b None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:34.820897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05386aa6-6acb-4557-a866-d84e7104dc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:34.821170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05386aa6-6acb-4557-a866-d84e7104dc8b None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:34.821842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05386aa6-6acb-4557-a866-d84e7104dc8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:34.822147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05386aa6-6acb-4557-a866-d84e7104dc8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:34.823544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05386aa6-6acb-4557-a866-d84e7104dc8b None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:34.834307 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 610/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:34.840142 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4556e-a52f-4839-8b3e-a6bf9ac3d05a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:34.849484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:34.945606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4556e-a52f-4839-8b3e-a6bf9ac3d05a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:34.979489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3d9ae-adef-4fe3-b6db-c8e4f28e7405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:34.980109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3d9ae-adef-4fe3-b6db-c8e4f28e7405 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:34.980392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3d9ae-adef-4fe3-b6db-c8e4f28e7405 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:34.980666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3d9ae-adef-4fe3-b6db-c8e4f28e7405 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:34.987270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3d9ae-adef-4fe3-b6db-c8e4f28e7405 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:34.988706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3d9ae-adef-4fe3-b6db-c8e4f28e7405 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-02f3d9ae-adef-4fe3-b6db-c8e4f28e7405 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:33:35.000418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:35.001478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:35.002427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:33:35.002427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:33:35.002427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:33:35.002427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:33:35.002427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:33:35.002427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:33:35.002427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:33:35.002427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:35.002427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 620/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:33:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:35.008222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05386aa6-6acb-4557-a866-d84e7104dc8b None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:35.105744 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05386aa6-6acb-4557-a866-d84e7104dc8b None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:35.140380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cc999-6974-419a-9f7f-b73a661814b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:35.140980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cc999-6974-419a-9f7f-b73a661814b5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:35.141260 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cc999-6974-419a-9f7f-b73a661814b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:35.141555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cc999-6974-419a-9f7f-b73a661814b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:35.142254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2cc999-6974-419a-9f7f-b73a661814b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:35.142563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2cc999-6974-419a-9f7f-b73a661814b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:35.143977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2cc999-6974-419a-9f7f-b73a661814b5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:35.157633 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 611/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:33:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:35.164232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3d9ae-adef-4fe3-b6db-c8e4f28e7405 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:35.229571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:35.264265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3d9ae-adef-4fe3-b6db-c8e4f28e7405 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:35.297514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548aeec-bafd-4653-aac0-8ba779af20cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:35.298189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548aeec-bafd-4653-aac0-8ba779af20cb None admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:35.298464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548aeec-bafd-4653-aac0-8ba779af20cb None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:35.298732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548aeec-bafd-4653-aac0-8ba779af20cb None admin] PATH_INFO: `/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/5e0aa6b215f646c280af245a89ccf849/roles/106b56dfde9d4c4f9f32ca4bec0b3e87` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:35.313202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548aeec-bafd-4653-aac0-8ba779af20cb None admin] RBAC: Authorizing `identity:create_grant(project_id=abcccf694e7b4f66a7372e0ede416caf, user_id=5e0aa6b215f646c280af245a89ccf849, role_id=106b56dfde9d4c4f9f32ca4bec0b3e87)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:35.314820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548aeec-bafd-4653-aac0-8ba779af20cb None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:35.452339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 621/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:33:35 2022] PUT /identity/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/5e0aa6b215f646c280af245a89ccf849/roles/106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:33:37.504041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416d521-788f-4b7f-ada8-01b350bc5399 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:37.504576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416d521-788f-4b7f-ada8-01b350bc5399 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:37.504576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416d521-788f-4b7f-ada8-01b350bc5399 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:37.506513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 612/1233] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:33:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:33:37.511601 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f53ec0-a444-4c6e-972b-f6d9f7ee072b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:37.511988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f53ec0-a444-4c6e-972b-f6d9f7ee072b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:37.512149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f53ec0-a444-4c6e-972b-f6d9f7ee072b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:37.530860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d5f53ec0-a444-4c6e-972b-f6d9f7ee072b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:37.532056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d5f53ec0-a444-4c6e-972b-f6d9f7ee072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:37.598295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 622/1234] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:33:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:33:37.621761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416d521-788f-4b7f-ada8-01b350bc5399 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:37.647549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:37.724566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416d521-788f-4b7f-ada8-01b350bc5399 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:37.758706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4d9d7-86e5-4db8-83ef-27033cc0f718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:37.759318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4d9d7-86e5-4db8-83ef-27033cc0f718 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:37.759599 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4d9d7-86e5-4db8-83ef-27033cc0f718 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:37.759873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4d9d7-86e5-4db8-83ef-27033cc0f718 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:37.769980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4d9d7-86e5-4db8-83ef-27033cc0f718 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-c6a4d9d7-86e5-4db8-83ef-27033cc0f718 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:33:37.774703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:37.783740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:37.784608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 613/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Tue Aug 9 18:33:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:37.785315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f53ec0-a444-4c6e-972b-f6d9f7ee072b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:37.828475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:37.879191 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f53ec0-a444-4c6e-972b-f6d9f7ee072b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:37.914031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23ee49-2635-4221-913b-477d7760198b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:37.914636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23ee49-2635-4221-913b-477d7760198b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:37.914917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23ee49-2635-4221-913b-477d7760198b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:37.915190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23ee49-2635-4221-913b-477d7760198b None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:37.915855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23ee49-2635-4221-913b-477d7760198b None admin] RBAC: Adding query filter params (name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:37.916157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23ee49-2635-4221-913b-477d7760198b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:37.917526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23ee49-2635-4221-913b-477d7760198b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:37.925493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 623/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Tue Aug 9 18:33:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:37.932357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4d9d7-86e5-4db8-83ef-27033cc0f718 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:38.014317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:38.029852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4d9d7-86e5-4db8-83ef-27033cc0f718 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:38.065198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1709e-5ec8-43e1-976c-912fe854915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:38.065848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1709e-5ec8-43e1-976c-912fe854915e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:38.066128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1709e-5ec8-43e1-976c-912fe854915e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:38.066400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1709e-5ec8-43e1-976c-912fe854915e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:38.076194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1709e-5ec8-43e1-976c-912fe854915e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:38.077658 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1709e-5ec8-43e1-976c-912fe854915e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-41a1709e-5ec8-43e1-976c-912fe854915e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:38.091609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:38.092716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:38.093694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:38.093694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:38.093694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:38.093694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 614/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:38.099170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23ee49-2635-4221-913b-477d7760198b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:38.205678 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23ee49-2635-4221-913b-477d7760198b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:38.214594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:38.239269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a5057-09ab-44ee-8b87-86660d5aa497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:38.240363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a5057-09ab-44ee-8b87-86660d5aa497 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:38.240743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a5057-09ab-44ee-8b87-86660d5aa497 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:38.241012 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a5057-09ab-44ee-8b87-86660d5aa497 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:38.241679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a5057-09ab-44ee-8b87-86660d5aa497 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:38.241981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a5057-09ab-44ee-8b87-86660d5aa497 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:38.243337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a5057-09ab-44ee-8b87-86660d5aa497 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:38.254936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 624/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:38.261004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1709e-5ec8-43e1-976c-912fe854915e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:38.360788 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1709e-5ec8-43e1-976c-912fe854915e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:38.394146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a4826-8762-43b2-92e0-5c241562fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:38.394756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a4826-8762-43b2-92e0-5c241562fb14 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:38.395029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a4826-8762-43b2-92e0-5c241562fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:38.395292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a4826-8762-43b2-92e0-5c241562fb14 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:38.398422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:38.402815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a4826-8762-43b2-92e0-5c241562fb14 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:38.406793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a4826-8762-43b2-92e0-5c241562fb14 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-391a4826-8762-43b2-92e0-5c241562fb14 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:33:38.415782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:38.416903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:38.418331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:38.418331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:38.418331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:33:38.418331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:33:38.418331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:33:38.418331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:33:38.418331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:33:38.418331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:33:38.418331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:33:38.418331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:38.418331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 615/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:38.422714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a5057-09ab-44ee-8b87-86660d5aa497 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:38.520128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a5057-09ab-44ee-8b87-86660d5aa497 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:38.554143 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdb10d-7438-4661-bc51-d3520dde8803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:38.554742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdb10d-7438-4661-bc51-d3520dde8803 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:38.555020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdb10d-7438-4661-bc51-d3520dde8803 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:38.555291 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdb10d-7438-4661-bc51-d3520dde8803 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:38.556013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdb10d-7438-4661-bc51-d3520dde8803 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:38.556324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdb10d-7438-4661-bc51-d3520dde8803 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:38.557769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdb10d-7438-4661-bc51-d3520dde8803 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:38.574711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 625/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Aug 9 18:33:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:38.580533 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a4826-8762-43b2-92e0-5c241562fb14 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:38.679840 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a4826-8762-43b2-92e0-5c241562fb14 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:38.712343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46030b-d438-4aef-9d45-08fc38518e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:38.712964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46030b-d438-4aef-9d45-08fc38518e6e None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:38.713243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46030b-d438-4aef-9d45-08fc38518e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:38.713512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46030b-d438-4aef-9d45-08fc38518e6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:38.773608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:38.779794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46030b-d438-4aef-9d45-08fc38518e6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:38.781221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46030b-d438-4aef-9d45-08fc38518e6e None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:38.876762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 616/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:33:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:33:38.883156 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdb10d-7438-4661-bc51-d3520dde8803 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:38.910712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:38.982075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdb10d-7438-4661-bc51-d3520dde8803 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:39.016387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90467281-c16c-410e-b2e0-b7fb80cabfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:39.017016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90467281-c16c-410e-b2e0-b7fb80cabfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:39.017302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90467281-c16c-410e-b2e0-b7fb80cabfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:39.017605 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90467281-c16c-410e-b2e0-b7fb80cabfd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:39.028300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90467281-c16c-410e-b2e0-b7fb80cabfd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:39.029792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90467281-c16c-410e-b2e0-b7fb80cabfd0 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-90467281-c16c-410e-b2e0-b7fb80cabfd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:39.044240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:39.045372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:39.046357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:39.046357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:39.046357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:39.046357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 626/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:39.051344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46030b-d438-4aef-9d45-08fc38518e6e None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:39.146026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:39.151793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46030b-d438-4aef-9d45-08fc38518e6e None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:39.185188 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d8313-81e9-4bb8-ab03-1c9124a5355f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:39.185818 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d8313-81e9-4bb8-ab03-1c9124a5355f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:39.186100 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d8313-81e9-4bb8-ab03-1c9124a5355f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:39.186374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d8313-81e9-4bb8-ab03-1c9124a5355f None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:39.187014 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d8313-81e9-4bb8-ab03-1c9124a5355f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:39.187318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d8313-81e9-4bb8-ab03-1c9124a5355f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:39.188717 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d8313-81e9-4bb8-ab03-1c9124a5355f None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:39.200797 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 617/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:39.206503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90467281-c16c-410e-b2e0-b7fb80cabfd0 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:39.286965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:39.308627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90467281-c16c-410e-b2e0-b7fb80cabfd0 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:39.342709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd49ed0-7287-4cbf-8dcc-a85cb87c71ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:39.343324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd49ed0-7287-4cbf-8dcc-a85cb87c71ca None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:39.343604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd49ed0-7287-4cbf-8dcc-a85cb87c71ca None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:39.343871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd49ed0-7287-4cbf-8dcc-a85cb87c71ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:39.350814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd49ed0-7287-4cbf-8dcc-a85cb87c71ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:39.352252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd49ed0-7287-4cbf-8dcc-a85cb87c71ca None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-ecd49ed0-7287-4cbf-8dcc-a85cb87c71ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:33:39.363727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:39.368679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:39.369590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:33:39.369590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:33:39.369590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:33:39.369590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:33:39.369590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:33:39.369590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:33:39.369590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:33:39.369590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:39.369590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 627/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:33:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:39.370711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d8313-81e9-4bb8-ab03-1c9124a5355f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:39.470256 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d8313-81e9-4bb8-ab03-1c9124a5355f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:39.502576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a65308-2f99-4084-ac56-c11043524c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:39.503179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a65308-2f99-4084-ac56-c11043524c73 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:39.503457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a65308-2f99-4084-ac56-c11043524c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:39.503753 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a65308-2f99-4084-ac56-c11043524c73 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:39.504437 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a65308-2f99-4084-ac56-c11043524c73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:39.505247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a65308-2f99-4084-ac56-c11043524c73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:39.506699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a65308-2f99-4084-ac56-c11043524c73 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:39.518801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 618/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:33:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:39.525752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd49ed0-7287-4cbf-8dcc-a85cb87c71ca None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:39.623810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd49ed0-7287-4cbf-8dcc-a85cb87c71ca None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:39.658155 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd648165-d0de-4050-9eca-b71951b48b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:39.658801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd648165-d0de-4050-9eca-b71951b48b7c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:39.659084 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd648165-d0de-4050-9eca-b71951b48b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:39.659354 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd648165-d0de-4050-9eca-b71951b48b7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:39.660080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd648165-d0de-4050-9eca-b71951b48b7c None admin] RBAC: Adding query filter params (role.id=106b56dfde9d4c4f9f32ca4bec0b3e87, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=5e0aa6b215f646c280af245a89ccf849) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:39.660390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd648165-d0de-4050-9eca-b71951b48b7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:39.661849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd648165-d0de-4050-9eca-b71951b48b7c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:39.667668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 628/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:33:39 2022] GET /identity/v3/role_assignments?user.id=5e0aa6b215f646c280af245a89ccf849&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=106b56dfde9d4c4f9f32ca4bec0b3e87 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:41.764098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2294a59-4b57-4233-b0e5-31a703bd4750 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:41.764672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2294a59-4b57-4233-b0e5-31a703bd4750 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:41.764672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2294a59-4b57-4233-b0e5-31a703bd4750 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:41.766402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 619/1247] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:33:41.771895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70503c41-5508-401b-a8c2-134f58acdbc6 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:41.772354 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70503c41-5508-401b-a8c2-134f58acdbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:41.772444 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70503c41-5508-401b-a8c2-134f58acdbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:41.778827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:41.791805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-70503c41-5508-401b-a8c2-134f58acdbc6 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:41.793108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-70503c41-5508-401b-a8c2-134f58acdbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:41.868097 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 629/1248] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:33:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:33:41.887282 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2294a59-4b57-4233-b0e5-31a703bd4750 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:41.905604 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:41.990449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2294a59-4b57-4233-b0e5-31a703bd4750 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:42.024431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b119fe-8113-41b4-ba79-97ff90f92e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:42.024995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b119fe-8113-41b4-ba79-97ff90f92e5f None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:42.025279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b119fe-8113-41b4-ba79-97ff90f92e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:42.025581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b119fe-8113-41b4-ba79-97ff90f92e5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:42.036732 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b119fe-8113-41b4-ba79-97ff90f92e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-56b119fe-8113-41b4-ba79-97ff90f92e5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:33:42.041462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:42.042635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:42.043710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 620/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:33:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:33:42.048743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70503c41-5508-401b-a8c2-134f58acdbc6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:42.155741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70503c41-5508-401b-a8c2-134f58acdbc6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:42.185396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:42.190688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b55c6e-9a33-43d6-9ddd-3ee74e3c5658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:42.191305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b55c6e-9a33-43d6-9ddd-3ee74e3c5658 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:42.191583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b55c6e-9a33-43d6-9ddd-3ee74e3c5658 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:42.191856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b55c6e-9a33-43d6-9ddd-3ee74e3c5658 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:42.192639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b55c6e-9a33-43d6-9ddd-3ee74e3c5658 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:42.193150 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b55c6e-9a33-43d6-9ddd-3ee74e3c5658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:42.195501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b55c6e-9a33-43d6-9ddd-3ee74e3c5658 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:42.214891 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 630/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:33:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:33:42.220740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b119fe-8113-41b4-ba79-97ff90f92e5f None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:42.301603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:42.325182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b119fe-8113-41b4-ba79-97ff90f92e5f None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:42.366694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc4ca2-342d-498a-8734-f94508c28ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:42.367403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc4ca2-342d-498a-8734-f94508c28ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:42.367738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc4ca2-342d-498a-8734-f94508c28ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:42.368069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc4ca2-342d-498a-8734-f94508c28ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:42.379485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc4ca2-342d-498a-8734-f94508c28ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:42.380991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc4ca2-342d-498a-8734-f94508c28ea3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-bdbc4ca2-342d-498a-8734-f94508c28ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:42.396216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:42.397424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:42.398534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:42.398534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:42.398534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:42.398534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 621/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:42.403061 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b55c6e-9a33-43d6-9ddd-3ee74e3c5658 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:42.510990 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b55c6e-9a33-43d6-9ddd-3ee74e3c5658 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:42.544771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bf1a1-bbb4-4635-89fe-0604b867040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:42.545372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bf1a1-bbb4-4635-89fe-0604b867040e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:42.545682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bf1a1-bbb4-4635-89fe-0604b867040e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:42.545954 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bf1a1-bbb4-4635-89fe-0604b867040e None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:42.546599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205bf1a1-bbb4-4635-89fe-0604b867040e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:42.546917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205bf1a1-bbb4-4635-89fe-0604b867040e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:42.548270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205bf1a1-bbb4-4635-89fe-0604b867040e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:42.560187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 631/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:42.565933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc4ca2-342d-498a-8734-f94508c28ea3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:42.665454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc4ca2-342d-498a-8734-f94508c28ea3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:42.690818 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:42.698936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8e5ff-cff9-42f1-aedf-d8ad02ebf6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:42.699550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8e5ff-cff9-42f1-aedf-d8ad02ebf6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:42.699829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8e5ff-cff9-42f1-aedf-d8ad02ebf6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:42.700099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8e5ff-cff9-42f1-aedf-d8ad02ebf6f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:42.707391 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8e5ff-cff9-42f1-aedf-d8ad02ebf6f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:42.709534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8e5ff-cff9-42f1-aedf-d8ad02ebf6f3 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-d8b8e5ff-cff9-42f1-aedf-d8ad02ebf6f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:33:42.720771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:42.721872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:42.723090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:42.723090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:42.723090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:33:42.723090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:33:42.723090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:33:42.723090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:33:42.723090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:33:42.723090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:33:42.723090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:33:42.723090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:42.723090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 622/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:42.727838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bf1a1-bbb4-4635-89fe-0604b867040e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:42.783408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:42.827912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bf1a1-bbb4-4635-89fe-0604b867040e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:42.861405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1facae-5633-4779-b9db-d3e232419914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:42.862026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1facae-5633-4779-b9db-d3e232419914 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:42.862304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1facae-5633-4779-b9db-d3e232419914 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:42.862573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1facae-5633-4779-b9db-d3e232419914 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:42.863283 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1facae-5633-4779-b9db-d3e232419914 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:42.863582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1facae-5633-4779-b9db-d3e232419914 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:42.864995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1facae-5633-4779-b9db-d3e232419914 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:42.882095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 632/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Aug 9 18:33:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:42.887971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8e5ff-cff9-42f1-aedf-d8ad02ebf6f3 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:42.990970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8e5ff-cff9-42f1-aedf-d8ad02ebf6f3 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:43.024938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9ffc3-52ae-4cbb-ac1e-22bf619b0922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:43.025573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9ffc3-52ae-4cbb-ac1e-22bf619b0922 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:43.025853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9ffc3-52ae-4cbb-ac1e-22bf619b0922 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:43.026132 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9ffc3-52ae-4cbb-ac1e-22bf619b0922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:43.082683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:43.094562 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9ffc3-52ae-4cbb-ac1e-22bf619b0922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:43.096018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9ffc3-52ae-4cbb-ac1e-22bf619b0922 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:43.200728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 623/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:33:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:33:43.206573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1facae-5633-4779-b9db-d3e232419914 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:43.308177 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1facae-5633-4779-b9db-d3e232419914 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:43.321795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:43.344156 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ad3b1-b7d0-47fc-b4a1-d2f38e04ab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:43.344699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ad3b1-b7d0-47fc-b4a1-d2f38e04ab5e None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:43.344979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ad3b1-b7d0-47fc-b4a1-d2f38e04ab5e None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:43.345255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ad3b1-b7d0-47fc-b4a1-d2f38e04ab5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:43.356194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ad3b1-b7d0-47fc-b4a1-d2f38e04ab5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:43.357676 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ad3b1-b7d0-47fc-b4a1-d2f38e04ab5e None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-dd8ad3b1-b7d0-47fc-b4a1-d2f38e04ab5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:43.373217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:43.374281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:43.375217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:43.375217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:43.375217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:43.375217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 633/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:43.380547 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9ffc3-52ae-4cbb-ac1e-22bf619b0922 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:43.476220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:43.488840 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9ffc3-52ae-4cbb-ac1e-22bf619b0922 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:43.523058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b8148-415d-4fda-9ee0-2053e8d5acd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:43.523652 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b8148-415d-4fda-9ee0-2053e8d5acd2 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:43.523931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b8148-415d-4fda-9ee0-2053e8d5acd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:43.524203 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b8148-415d-4fda-9ee0-2053e8d5acd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:43.524843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b8148-415d-4fda-9ee0-2053e8d5acd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:43.525149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b8148-415d-4fda-9ee0-2053e8d5acd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:43.526543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b8148-415d-4fda-9ee0-2053e8d5acd2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:43.538783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 624/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:43.544741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ad3b1-b7d0-47fc-b4a1-d2f38e04ab5e None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:43.649007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ad3b1-b7d0-47fc-b4a1-d2f38e04ab5e None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:43.682777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f834e20-3042-4017-9166-f7025e799e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:43.683391 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f834e20-3042-4017-9166-f7025e799e3c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:43.683667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f834e20-3042-4017-9166-f7025e799e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:43.683935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f834e20-3042-4017-9166-f7025e799e3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:43.690990 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f834e20-3042-4017-9166-f7025e799e3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:43.692468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f834e20-3042-4017-9166-f7025e799e3c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-7f834e20-3042-4017-9166-f7025e799e3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:33:43.703681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:43.704724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:43.711095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:33:43.711095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:33:43.711095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:33:43.711095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:33:43.711095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:33:43.711095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:33:43.711095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:33:43.711095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:43.711095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 634/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:33:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:43.711679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b8148-415d-4fda-9ee0-2053e8d5acd2 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:43.817515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b8148-415d-4fda-9ee0-2053e8d5acd2 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:43.851384 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264bfa9-62b9-48de-8154-0d27d0eeab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:43.851983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bfa9-62b9-48de-8154-0d27d0eeab33 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:43.852269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bfa9-62b9-48de-8154-0d27d0eeab33 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:43.852543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bfa9-62b9-48de-8154-0d27d0eeab33 None admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:43.853234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264bfa9-62b9-48de-8154-0d27d0eeab33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:43.853560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264bfa9-62b9-48de-8154-0d27d0eeab33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:43.854993 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264bfa9-62b9-48de-8154-0d27d0eeab33 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:43.867410 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 625/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:43.873819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f834e20-3042-4017-9166-f7025e799e3c None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:43.899984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:43.974827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f834e20-3042-4017-9166-f7025e799e3c None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:44.008204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdaa94d-5ba5-41fb-978d-dcb7e50f2706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:44.008808 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaa94d-5ba5-41fb-978d-dcb7e50f2706 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:44.009091 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaa94d-5ba5-41fb-978d-dcb7e50f2706 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:44.009368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaa94d-5ba5-41fb-978d-dcb7e50f2706 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:44.010128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdaa94d-5ba5-41fb-978d-dcb7e50f2706 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=5e0aa6b215f646c280af245a89ccf849) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:44.012640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdaa94d-5ba5-41fb-978d-dcb7e50f2706 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:44.012640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdaa94d-5ba5-41fb-978d-dcb7e50f2706 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:44.017189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 635/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:33:43 2022] GET /identity/v3/role_assignments?user.id=5e0aa6b215f646c280af245a89ccf849&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:45.999150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827bf8-914b-43d3-a051-2fe12194db4b None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:45.999734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827bf8-914b-43d3-a051-2fe12194db4b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:45.999734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827bf8-914b-43d3-a051-2fe12194db4b None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:46.001017 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 626/1261] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:33:46.006580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a7b91-85ef-4cf1-ad77-953e9ca96ad7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:46.006979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a7b91-85ef-4cf1-ad77-953e9ca96ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:46.007125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a7b91-85ef-4cf1-ad77-953e9ca96ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:46.024617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-3c3a7b91-85ef-4cf1-ad77-953e9ca96ad7 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:46.025801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3c3a7b91-85ef-4cf1-ad77-953e9ca96ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:46.094800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 636/1262] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:33:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:33:46.117479 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79827bf8-914b-43d3-a051-2fe12194db4b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:46.128236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:46.216168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79827bf8-914b-43d3-a051-2fe12194db4b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:46.257191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c31309-6714-4c1e-8090-e2beb45eef2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:46.257760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c31309-6714-4c1e-8090-e2beb45eef2c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:46.258040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c31309-6714-4c1e-8090-e2beb45eef2c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:46.258310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c31309-6714-4c1e-8090-e2beb45eef2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:46.269505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c31309-6714-4c1e-8090-e2beb45eef2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-73c31309-6714-4c1e-8090-e2beb45eef2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:33:46.273731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:46.274867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:46.276025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 627/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:33:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:46.280860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a7b91-85ef-4cf1-ad77-953e9ca96ad7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:46.303041 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:46.381351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a7b91-85ef-4cf1-ad77-953e9ca96ad7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:46.414644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15e209-c16f-4b02-b917-9aec48b6e139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:46.415240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15e209-c16f-4b02-b917-9aec48b6e139 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:46.415519 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15e209-c16f-4b02-b917-9aec48b6e139 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:46.415789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15e209-c16f-4b02-b917-9aec48b6e139 None admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:46.416483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15e209-c16f-4b02-b917-9aec48b6e139 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:46.416786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15e209-c16f-4b02-b917-9aec48b6e139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:46.418175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15e209-c16f-4b02-b917-9aec48b6e139 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:46.426058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 637/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:33:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:46.432786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c31309-6714-4c1e-8090-e2beb45eef2c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:46.501767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:46.536854 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c31309-6714-4c1e-8090-e2beb45eef2c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:46.570112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8491a2f-37fc-4c5d-a902-fad3eb7840f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:46.570736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8491a2f-37fc-4c5d-a902-fad3eb7840f8 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:46.571016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8491a2f-37fc-4c5d-a902-fad3eb7840f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:46.571380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8491a2f-37fc-4c5d-a902-fad3eb7840f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:46.581538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8491a2f-37fc-4c5d-a902-fad3eb7840f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:46.582988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8491a2f-37fc-4c5d-a902-fad3eb7840f8 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-c8491a2f-37fc-4c5d-a902-fad3eb7840f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:46.597156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:46.598815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:46.600196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:46.600196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:46.600196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:46.600196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 628/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:46.604561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15e209-c16f-4b02-b917-9aec48b6e139 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:46.686593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:46.704596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15e209-c16f-4b02-b917-9aec48b6e139 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:46.737675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c3e73-5c06-451d-9e73-64734ce580c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:46.738278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c3e73-5c06-451d-9e73-64734ce580c6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:46.738559 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c3e73-5c06-451d-9e73-64734ce580c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:46.738887 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c3e73-5c06-451d-9e73-64734ce580c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:46.739520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c3e73-5c06-451d-9e73-64734ce580c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:46.739871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c3e73-5c06-451d-9e73-64734ce580c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:46.741234 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c3e73-5c06-451d-9e73-64734ce580c6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:46.752106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 638/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:46.757887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8491a2f-37fc-4c5d-a902-fad3eb7840f8 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:46.856811 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8491a2f-37fc-4c5d-a902-fad3eb7840f8 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:46.878242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:46.892188 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b00d-397a-4db8-bcfc-685c959545e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:46.892810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b00d-397a-4db8-bcfc-685c959545e5 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:46.893090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b00d-397a-4db8-bcfc-685c959545e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:46.893362 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b00d-397a-4db8-bcfc-685c959545e5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:46.901181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8b00d-397a-4db8-bcfc-685c959545e5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:46.902639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8b00d-397a-4db8-bcfc-685c959545e5 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-16e8b00d-397a-4db8-bcfc-685c959545e5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:33:46.914498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:46.920214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:46.921457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:46.921457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:46.921457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:33:46.921457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:33:46.921457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:33:46.921457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:33:46.921457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:33:46.921457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:33:46.921457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:33:46.921457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:46.921457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 629/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:33:46.923254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c3e73-5c06-451d-9e73-64734ce580c6 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:47.027046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c3e73-5c06-451d-9e73-64734ce580c6 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:47.059697 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d0c9c-b194-489b-b3d2-ad60f263b329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:47.060295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d0c9c-b194-489b-b3d2-ad60f263b329 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:47.060575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d0c9c-b194-489b-b3d2-ad60f263b329 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:47.060844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d0c9c-b194-489b-b3d2-ad60f263b329 None admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:47.061602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d0c9c-b194-489b-b3d2-ad60f263b329 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:47.061915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d0c9c-b194-489b-b3d2-ad60f263b329 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:47.063341 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d0c9c-b194-489b-b3d2-ad60f263b329 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:47.072777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:47.080811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 639/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Aug 9 18:33:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:47.087203 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b00d-397a-4db8-bcfc-685c959545e5 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:47.193834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b00d-397a-4db8-bcfc-685c959545e5 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:47.227754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60a73e-9a3f-43ba-88f7-bab05c500038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:47.228378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60a73e-9a3f-43ba-88f7-bab05c500038 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:47.228661 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60a73e-9a3f-43ba-88f7-bab05c500038 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:47.228935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60a73e-9a3f-43ba-88f7-bab05c500038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:47.239478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60a73e-9a3f-43ba-88f7-bab05c500038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:47.240953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60a73e-9a3f-43ba-88f7-bab05c500038 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-ad60a73e-9a3f-43ba-88f7-bab05c500038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:47.255350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:47.260942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:47.262084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:47.262084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:47.262084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:47.262084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 630/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:47.263317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d0c9c-b194-489b-b3d2-ad60f263b329 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:47.362332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d0c9c-b194-489b-b3d2-ad60f263b329 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:47.396993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6363ddc-5fcd-476b-84c0-ed80e3053e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:47.397621 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6363ddc-5fcd-476b-84c0-ed80e3053e7a None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:47.397903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6363ddc-5fcd-476b-84c0-ed80e3053e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:47.398175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6363ddc-5fcd-476b-84c0-ed80e3053e7a None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:47.398843 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6363ddc-5fcd-476b-84c0-ed80e3053e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:47.399143 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6363ddc-5fcd-476b-84c0-ed80e3053e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:47.400524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6363ddc-5fcd-476b-84c0-ed80e3053e7a None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:47.411590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 640/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:47.417480 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60a73e-9a3f-43ba-88f7-bab05c500038 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:47.449916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:47.515461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60a73e-9a3f-43ba-88f7-bab05c500038 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:47.548620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6c343-46b6-461d-9187-a15d4952ed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:47.549238 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6c343-46b6-461d-9187-a15d4952ed71 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:47.549516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6c343-46b6-461d-9187-a15d4952ed71 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:47.549819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6c343-46b6-461d-9187-a15d4952ed71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:47.557030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6c343-46b6-461d-9187-a15d4952ed71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:47.558489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6c343-46b6-461d-9187-a15d4952ed71 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-c8f6c343-46b6-461d-9187-a15d4952ed71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:33:47.570189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:47.571077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:47.572013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:33:47.572013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:33:47.572013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:33:47.572013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:33:47.572013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:33:47.572013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:33:47.572013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:33:47.572013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:47.572013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 631/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:33:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:47.577120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6363ddc-5fcd-476b-84c0-ed80e3053e7a None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:47.635488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:47.678265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6363ddc-5fcd-476b-84c0-ed80e3053e7a None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:47.713571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4f274-e2b4-4d41-ac65-16530a1ee17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:47.714102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4f274-e2b4-4d41-ac65-16530a1ee17c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:47.714379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4f274-e2b4-4d41-ac65-16530a1ee17c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:47.714647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4f274-e2b4-4d41-ac65-16530a1ee17c None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:47.715400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4f274-e2b4-4d41-ac65-16530a1ee17c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:47.715702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4f274-e2b4-4d41-ac65-16530a1ee17c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:47.717128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4f274-e2b4-4d41-ac65-16530a1ee17c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:47.730658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 641/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:33:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:47.736750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6c343-46b6-461d-9187-a15d4952ed71 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:47.827397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:47.835729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6c343-46b6-461d-9187-a15d4952ed71 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:47.870095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702ac4e-814d-467e-a859-da0adf849fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:47.870750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702ac4e-814d-467e-a859-da0adf849fe2 None admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:47.871032 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702ac4e-814d-467e-a859-da0adf849fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:47.871315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702ac4e-814d-467e-a859-da0adf849fe2 None admin] PATH_INFO: `/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/5e0aa6b215f646c280af245a89ccf849/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:47.873197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702ac4e-814d-467e-a859-da0adf849fe2 None admin] RBAC: Authorizing `identity:create_grant(project_id=abcccf694e7b4f66a7372e0ede416caf, user_id=5e0aa6b215f646c280af245a89ccf849, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:47.874701 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702ac4e-814d-467e-a859-da0adf849fe2 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:48.017905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 632/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Tue Aug 9 18:33:47 2022] PUT /identity/v3/projects/abcccf694e7b4f66a7372e0ede416caf/users/5e0aa6b215f646c280af245a89ccf849/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:33:50.001513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027072d-ff5f-47fe-9ea5-a32aa84cb8b9 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:50.002066 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027072d-ff5f-47fe-9ea5-a32aa84cb8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:50.002159 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027072d-ff5f-47fe-9ea5-a32aa84cb8b9 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:50.003461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 642/1274] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:33:50.009078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466e21c-abc0-43fe-9a6d-bf2ede9dfbd1 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:50.009477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466e21c-abc0-43fe-9a6d-bf2ede9dfbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:50.009654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466e21c-abc0-43fe-9a6d-bf2ede9dfbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:50.027364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a466e21c-abc0-43fe-9a6d-bf2ede9dfbd1 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:50.028639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a466e21c-abc0-43fe-9a6d-bf2ede9dfbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:50.093830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 633/1275] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:33:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:33:50.118412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027072d-ff5f-47fe-9ea5-a32aa84cb8b9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:50.228422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027072d-ff5f-47fe-9ea5-a32aa84cb8b9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:50.246062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:50.262674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137f46f-84f0-4658-9399-11272ec3dc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:50.263214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137f46f-84f0-4658-9399-11272ec3dc09 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:50.263496 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137f46f-84f0-4658-9399-11272ec3dc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:50.263767 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137f46f-84f0-4658-9399-11272ec3dc09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:50.274007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137f46f-84f0-4658-9399-11272ec3dc09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-b137f46f-84f0-4658-9399-11272ec3dc09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:33:50.278759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:50.279858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:50.280748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 643/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Tue Aug 9 18:33:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:33:50.286670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466e21c-abc0-43fe-9a6d-bf2ede9dfbd1 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:50.368167 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:50.385803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466e21c-abc0-43fe-9a6d-bf2ede9dfbd1 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:50.419034 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0b3d0-b4bd-42ca-b663-ba7be2559c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:50.419645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0b3d0-b4bd-42ca-b663-ba7be2559c77 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:50.419927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0b3d0-b4bd-42ca-b663-ba7be2559c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:50.420202 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0b3d0-b4bd-42ca-b663-ba7be2559c77 None admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:50.420868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0b3d0-b4bd-42ca-b663-ba7be2559c77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:50.421175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0b3d0-b4bd-42ca-b663-ba7be2559c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:50.422634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0b3d0-b4bd-42ca-b663-ba7be2559c77 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:50.432126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 634/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Tue Aug 9 18:33:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:50.438021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137f46f-84f0-4658-9399-11272ec3dc09 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:50.536848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137f46f-84f0-4658-9399-11272ec3dc09 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:50.571364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59eae1a-e1d7-4828-9dcf-65f999be2c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:50.571992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59eae1a-e1d7-4828-9dcf-65f999be2c6b None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:50.572276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59eae1a-e1d7-4828-9dcf-65f999be2c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:50.572549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59eae1a-e1d7-4828-9dcf-65f999be2c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:50.582678 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59eae1a-e1d7-4828-9dcf-65f999be2c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:50.585618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59eae1a-e1d7-4828-9dcf-65f999be2c6b None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-f59eae1a-e1d7-4828-9dcf-65f999be2c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:50.598692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:50.599783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:50.600849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:50.600849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:50.600849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:50.600849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 644/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:50.605646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0b3d0-b4bd-42ca-b663-ba7be2559c77 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:50.704693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0b3d0-b4bd-42ca-b663-ba7be2559c77 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:50.737326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867b216-3e9d-489d-bbd8-30c822ee5cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:50.737936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867b216-3e9d-489d-bbd8-30c822ee5cda None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:50.738210 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867b216-3e9d-489d-bbd8-30c822ee5cda None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:50.738479 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867b216-3e9d-489d-bbd8-30c822ee5cda None admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:50.739630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867b216-3e9d-489d-bbd8-30c822ee5cda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:50.739929 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867b216-3e9d-489d-bbd8-30c822ee5cda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:50.741501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867b216-3e9d-489d-bbd8-30c822ee5cda None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:50.753033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 635/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:50.759337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59eae1a-e1d7-4828-9dcf-65f999be2c6b None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:50.789815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:50.857699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59eae1a-e1d7-4828-9dcf-65f999be2c6b None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:50.893836 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45d32b-e2bf-4d64-800e-910604f994ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:50.894453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45d32b-e2bf-4d64-800e-910604f994ba None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:50.894730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45d32b-e2bf-4d64-800e-910604f994ba None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:50.894998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45d32b-e2bf-4d64-800e-910604f994ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:50.902533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45d32b-e2bf-4d64-800e-910604f994ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:50.903961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45d32b-e2bf-4d64-800e-910604f994ba None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-fb45d32b-e2bf-4d64-800e-910604f994ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 18:33:50.915239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:50.916366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:50.917733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:50.917733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:50.917733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 18:33:50.917733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 18:33:50.917733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 18:33:50.917733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 18:33:50.917733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 18:33:50.917733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 18:33:50.917733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 18:33:50.917733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:50.917733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 645/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:50.922192 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867b216-3e9d-489d-bbd8-30c822ee5cda None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:50.926007 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:51.019553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867b216-3e9d-489d-bbd8-30c822ee5cda None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:51.056710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406af396-e058-42d6-882d-9028baf44e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:51.057298 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406af396-e058-42d6-882d-9028baf44e2c None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:51.057679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406af396-e058-42d6-882d-9028baf44e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:51.057997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406af396-e058-42d6-882d-9028baf44e2c None admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:51.058777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406af396-e058-42d6-882d-9028baf44e2c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:51.059129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406af396-e058-42d6-882d-9028baf44e2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:51.060779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406af396-e058-42d6-882d-9028baf44e2c None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:51.078865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 636/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Tue Aug 9 18:33:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:51.085323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45d32b-e2bf-4d64-800e-910604f994ba None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:51.180138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:51.189291 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45d32b-e2bf-4d64-800e-910604f994ba None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:51.222617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e91886-d8f8-4f5e-b6a0-6756088f0db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:51.224911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e91886-d8f8-4f5e-b6a0-6756088f0db7 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:51.224911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e91886-d8f8-4f5e-b6a0-6756088f0db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:51.224911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e91886-d8f8-4f5e-b6a0-6756088f0db7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:51.292569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e91886-d8f8-4f5e-b6a0-6756088f0db7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:51.294082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e91886-d8f8-4f5e-b6a0-6756088f0db7 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:51.381385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:51.391223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 646/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Tue Aug 9 18:33:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 18:33:51.396925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406af396-e058-42d6-882d-9028baf44e2c None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:51.464937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:51.501756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406af396-e058-42d6-882d-9028baf44e2c None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:51.534421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93eb26-df76-4512-aa7d-e4c93b074159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:51.535116 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93eb26-df76-4512-aa7d-e4c93b074159 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:51.536168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93eb26-df76-4512-aa7d-e4c93b074159 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:51.536474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93eb26-df76-4512-aa7d-e4c93b074159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:51.546896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93eb26-df76-4512-aa7d-e4c93b074159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:51.548374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93eb26-df76-4512-aa7d-e4c93b074159 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-4d93eb26-df76-4512-aa7d-e4c93b074159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:33:51.563494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:51.566021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:33:51.567152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:33:51.567152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:33:51.567152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:51.567152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 637/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:51.570385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e91886-d8f8-4f5e-b6a0-6756088f0db7 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:51.670573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e91886-d8f8-4f5e-b6a0-6756088f0db7 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:51.704148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f6c4c-2e23-4b53-aac2-f2d283ed5196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:51.704752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f6c4c-2e23-4b53-aac2-f2d283ed5196 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:51.705032 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f6c4c-2e23-4b53-aac2-f2d283ed5196 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:51.705308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f6c4c-2e23-4b53-aac2-f2d283ed5196 None admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:51.705988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f6c4c-2e23-4b53-aac2-f2d283ed5196 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:51.706292 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f6c4c-2e23-4b53-aac2-f2d283ed5196 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:51.707677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f6c4c-2e23-4b53-aac2-f2d283ed5196 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:51.718948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 647/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Tue Aug 9 18:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:51.724972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93eb26-df76-4512-aa7d-e4c93b074159 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:51.827329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93eb26-df76-4512-aa7d-e4c93b074159 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:51.840776 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:51.860386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220610c-b60d-44b9-a4b1-34c6d6c87bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:51.860994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220610c-b60d-44b9-a4b1-34c6d6c87bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:51.861268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220610c-b60d-44b9-a4b1-34c6d6c87bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:51.861554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220610c-b60d-44b9-a4b1-34c6d6c87bf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:51.868498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220610c-b60d-44b9-a4b1-34c6d6c87bf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:51.869951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220610c-b60d-44b9-a4b1-34c6d6c87bf4 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application [None req-b220610c-b60d-44b9-a4b1-34c6d6c87bf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 18:33:51.880672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:51.881738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:51.882630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 18:33:51.882630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 18:33:51.882630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 18:33:51.882630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 18:33:51.882630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 18:33:51.882630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 18:33:51.882630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 18:33:51.882630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: ERROR keystone.server.flask.application Aug 09 18:33:51.882630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 638/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Tue Aug 9 18:33:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:51.887771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f6c4c-2e23-4b53-aac2-f2d283ed5196 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:51.937959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:51.987834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f6c4c-2e23-4b53-aac2-f2d283ed5196 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:52.020854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257c01b-21e0-4b91-85c9-4e5b7b439d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:52.021446 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257c01b-21e0-4b91-85c9-4e5b7b439d62 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:52.021839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257c01b-21e0-4b91-85c9-4e5b7b439d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:52.022043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257c01b-21e0-4b91-85c9-4e5b7b439d62 None admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:52.022752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257c01b-21e0-4b91-85c9-4e5b7b439d62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:52.023263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257c01b-21e0-4b91-85c9-4e5b7b439d62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:52.025003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257c01b-21e0-4b91-85c9-4e5b7b439d62 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:52.038623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 648/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Tue Aug 9 18:33:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:33:52.044854 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220610c-b60d-44b9-a4b1-34c6d6c87bf4 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:52.151818 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220610c-b60d-44b9-a4b1-34c6d6c87bf4 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:52.190497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c0be1-2fb8-4572-a5dd-6cbf21b1eaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:52.191088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c0be1-2fb8-4572-a5dd-6cbf21b1eaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:52.191371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c0be1-2fb8-4572-a5dd-6cbf21b1eaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:52.191648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c0be1-2fb8-4572-a5dd-6cbf21b1eaa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:52.192428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c0be1-2fb8-4572-a5dd-6cbf21b1eaa1 None admin] RBAC: Adding query filter params (role.id=243b2ac105fc4306afd7108a6cdb0982, scope.project.id=abcccf694e7b4f66a7372e0ede416caf, user.id=5e0aa6b215f646c280af245a89ccf849) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:52.192740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c0be1-2fb8-4572-a5dd-6cbf21b1eaa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:52.194300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c0be1-2fb8-4572-a5dd-6cbf21b1eaa1 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:52.200943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 639/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Tue Aug 9 18:33:52 2022] GET /identity/v3/role_assignments?user.id=5e0aa6b215f646c280af245a89ccf849&scope.project.id=abcccf694e7b4f66a7372e0ede416caf&role.id=243b2ac105fc4306afd7108a6cdb0982 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:54.302981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd57bc-5722-4c69-b8d7-b15c2b53a4e1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:54.303463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd57bc-5722-4c69-b8d7-b15c2b53a4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:54.303580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd57bc-5722-4c69-b8d7-b15c2b53a4e1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:54.304878 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 649/1288] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:33:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:33:54.310232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e82bcb-ba56-4cb3-9c35-aa23c5669c73 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:54.310659 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e82bcb-ba56-4cb3-9c35-aa23c5669c73 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:54.310782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e82bcb-ba56-4cb3-9c35-aa23c5669c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:54.336706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-71e82bcb-ba56-4cb3-9c35-aa23c5669c73 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:54.338959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-71e82bcb-ba56-4cb3-9c35-aa23c5669c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:54.369830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:54.404948 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 640/1289] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:33:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:33:54.427401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd57bc-5722-4c69-b8d7-b15c2b53a4e1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:54.533950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd57bc-5722-4c69-b8d7-b15c2b53a4e1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:54.545785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:54.569634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5ed0f-ca27-46a6-bc44-234412ff5222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:54.570259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5ed0f-ca27-46a6-bc44-234412ff5222 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:54.570540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5ed0f-ca27-46a6-bc44-234412ff5222 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:54.570814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5ed0f-ca27-46a6-bc44-234412ff5222 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:54.571397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5ed0f-ca27-46a6-bc44-234412ff5222 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:54.572774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5ed0f-ca27-46a6-bc44-234412ff5222 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-82f5ed0f-ca27-46a6-bc44-234412ff5222 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:54.581255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:54.582548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:54.583417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 650/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 9 18:33:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:54.588426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e82bcb-ba56-4cb3-9c35-aa23c5669c73 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:54.687687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e82bcb-ba56-4cb3-9c35-aa23c5669c73 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:54.721389 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2017f-e44c-48c2-b806-17e269511344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:54.722031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2017f-e44c-48c2-b806-17e269511344 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:54.722315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2017f-e44c-48c2-b806-17e269511344 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:54.722587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2017f-e44c-48c2-b806-17e269511344 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:54.723745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2017f-e44c-48c2-b806-17e269511344 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:54.724138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2017f-e44c-48c2-b806-17e269511344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:54.725494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2017f-e44c-48c2-b806-17e269511344 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:54.730191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 641/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Tue Aug 9 18:33:54 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:54.735940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5ed0f-ca27-46a6-bc44-234412ff5222 None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:54.833277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5ed0f-ca27-46a6-bc44-234412ff5222 None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:54.866737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d2423-2527-417b-be3a-ebb6ee98bc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:54.867319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d2423-2527-417b-be3a-ebb6ee98bc2c None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:54.867600 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d2423-2527-417b-be3a-ebb6ee98bc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:54.867863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d2423-2527-417b-be3a-ebb6ee98bc2c None admin] PATH_INFO: `/v3/services` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:54.868482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d2423-2527-417b-be3a-ebb6ee98bc2c None admin] RBAC: Adding query filter params (type=placement) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:54.868781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d2423-2527-417b-be3a-ebb6ee98bc2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:54.870158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d2423-2527-417b-be3a-ebb6ee98bc2c None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:54.874545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 651/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Tue Aug 9 18:33:54 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:56.877976 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f9009-6d69-4d12-8b90-2613dad01229 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:56.878508 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f9009-6d69-4d12-8b90-2613dad01229 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:56.878508 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f9009-6d69-4d12-8b90-2613dad01229 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:56.880431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 642/1293] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:33:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:33:56.885876 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef87c77-e599-4f54-9cbd-fe0acaaa4457 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:56.886290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef87c77-e599-4f54-9cbd-fe0acaaa4457 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:56.886429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef87c77-e599-4f54-9cbd-fe0acaaa4457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:56.904700 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5ef87c77-e599-4f54-9cbd-fe0acaaa4457 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:56.907847 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5ef87c77-e599-4f54-9cbd-fe0acaaa4457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:56.936096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:56.973924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 652/1294] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:33:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:33:56.996353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f9009-6d69-4d12-8b90-2613dad01229 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:57.028409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:57.098792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f9009-6d69-4d12-8b90-2613dad01229 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:57.132959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34148b60-93be-4dcd-92ce-131e535cfd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:57.133483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34148b60-93be-4dcd-92ce-131e535cfd52 None admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:57.133799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34148b60-93be-4dcd-92ce-131e535cfd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:57.134076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34148b60-93be-4dcd-92ce-131e535cfd52 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:57.134719 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34148b60-93be-4dcd-92ce-131e535cfd52 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:57.136075 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34148b60-93be-4dcd-92ce-131e535cfd52 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:57.281027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 643/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Tue Aug 9 18:33:56 2022] POST /identity/v3/services => generated 246 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:33:59.307065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede32e59-3e24-45dc-b7a3-a4b8666b9e29 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:59.307558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede32e59-3e24-45dc-b7a3-a4b8666b9e29 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:59.307650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede32e59-3e24-45dc-b7a3-a4b8666b9e29 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:59.308948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 653/1296] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:33:59.314100 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50affd6c-635f-4b80-a379-53a1a3e2c278 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:59.314559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50affd6c-635f-4b80-a379-53a1a3e2c278 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:59.314680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50affd6c-635f-4b80-a379-53a1a3e2c278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:59.333120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-50affd6c-635f-4b80-a379-53a1a3e2c278 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:33:59.334338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-50affd6c-635f-4b80-a379-53a1a3e2c278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:33:59.406812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 644/1297] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:33:59 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:33:59.430300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede32e59-3e24-45dc-b7a3-a4b8666b9e29 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:59.544874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede32e59-3e24-45dc-b7a3-a4b8666b9e29 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:59.578494 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19beeab2-636b-402b-9a40-51f257a13c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:59.579106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beeab2-636b-402b-9a40-51f257a13c6d None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:59.579414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beeab2-636b-402b-9a40-51f257a13c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:59.579658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beeab2-636b-402b-9a40-51f257a13c6d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:59.580247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19beeab2-636b-402b-9a40-51f257a13c6d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:59.581638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19beeab2-636b-402b-9a40-51f257a13c6d None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-19beeab2-636b-402b-9a40-51f257a13c6d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:33:59.590805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:33:59.592000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:33:59.593001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 654/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 9 18:33:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:33:59.598646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50affd6c-635f-4b80-a379-53a1a3e2c278 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:59.616688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:59.695050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50affd6c-635f-4b80-a379-53a1a3e2c278 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:59.728972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3994ac5-9abd-4306-a735-d504dd3fed67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:59.729594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3994ac5-9abd-4306-a735-d504dd3fed67 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:59.729878 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3994ac5-9abd-4306-a735-d504dd3fed67 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:59.730162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3994ac5-9abd-4306-a735-d504dd3fed67 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:59.730785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3994ac5-9abd-4306-a735-d504dd3fed67 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:59.731089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3994ac5-9abd-4306-a735-d504dd3fed67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:59.732448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3994ac5-9abd-4306-a735-d504dd3fed67 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:59.737458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 645/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Tue Aug 9 18:33:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:59.743631 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19beeab2-636b-402b-9a40-51f257a13c6d None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:59.776952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:33:59.845319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19beeab2-636b-402b-9a40-51f257a13c6d None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:33:59.894541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7affe-6d68-4dd9-a872-8d62f2ef28b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:33:59.894541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7affe-6d68-4dd9-a872-8d62f2ef28b6 None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:33:59.894541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7affe-6d68-4dd9-a872-8d62f2ef28b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:33:59.894541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7affe-6d68-4dd9-a872-8d62f2ef28b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:33:59.895002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7affe-6d68-4dd9-a872-8d62f2ef28b6 None admin] RBAC: Adding query filter params (interface=public, service_id=f5d2aed9dc73416782c4630af2ac0e7e, region_id=RegionOne) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:33:59.895002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7affe-6d68-4dd9-a872-8d62f2ef28b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:33:59.895002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7affe-6d68-4dd9-a872-8d62f2ef28b6 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:33:59.898257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 655/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Tue Aug 9 18:33:59 2022] GET /identity/v3/endpoints?service_id=f5d2aed9dc73416782c4630af2ac0e7e&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:33:59.903982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3994ac5-9abd-4306-a735-d504dd3fed67 None admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:33:59.984340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:00.001397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3994ac5-9abd-4306-a735-d504dd3fed67 None admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:00.035289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29986b-77b6-46bf-ba1b-a2c6db794a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:00.035886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29986b-77b6-46bf-ba1b-a2c6db794a51 None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:00.036166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29986b-77b6-46bf-ba1b-a2c6db794a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:00.036438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29986b-77b6-46bf-ba1b-a2c6db794a51 None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:00.037051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29986b-77b6-46bf-ba1b-a2c6db794a51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:00.038519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29986b-77b6-46bf-ba1b-a2c6db794a51 None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:00.045859 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 646/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Tue Aug 9 18:33:59 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:34:02.047835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603182dd-ecbc-4f48-8a28-f32da70fe4d5 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:02.048425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603182dd-ecbc-4f48-8a28-f32da70fe4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:02.048425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603182dd-ecbc-4f48-8a28-f32da70fe4d5 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:02.049763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 656/1302] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:34:02.055204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4119b-2b44-4fde-895c-fc6d0647db25 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:02.055598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4119b-2b44-4fde-895c-fc6d0647db25 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:02.055708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4119b-2b44-4fde-895c-fc6d0647db25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:02.073905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-cbb4119b-2b44-4fde-895c-fc6d0647db25 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:34:02.075092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cbb4119b-2b44-4fde-895c-fc6d0647db25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:34:02.142142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 647/1303] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:34:02 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 18:34:02.166744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603182dd-ecbc-4f48-8a28-f32da70fe4d5 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:02.262185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:02.268859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603182dd-ecbc-4f48-8a28-f32da70fe4d5 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:02.303282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7b6e6-2373-4499-8e15-9868eb1c85fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:02.303871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7b6e6-2373-4499-8e15-9868eb1c85fd None admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:02.304154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7b6e6-2373-4499-8e15-9868eb1c85fd None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:02.304425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7b6e6-2373-4499-8e15-9868eb1c85fd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:02.305050 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a7b6e6-2373-4499-8e15-9868eb1c85fd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:02.306511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a7b6e6-2373-4499-8e15-9868eb1c85fd None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application [None req-c6a7b6e6-2373-4499-8e15-9868eb1c85fd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:34:02.315173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:34:02.317054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application with Lock( Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: ERROR keystone.server.flask.application Aug 09 18:34:02.318353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 657/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Tue Aug 9 18:34:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:34:02.322974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4119b-2b44-4fde-895c-fc6d0647db25 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:02.400390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:02.422113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4119b-2b44-4fde-895c-fc6d0647db25 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:02.461996 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d184514-c846-4a9e-a188-d4103fc2cb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:02.462850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d184514-c846-4a9e-a188-d4103fc2cb8a None admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:02.463290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d184514-c846-4a9e-a188-d4103fc2cb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:02.463961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d184514-c846-4a9e-a188-d4103fc2cb8a None admin] PATH_INFO: `/v3/services` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:02.464621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d184514-c846-4a9e-a188-d4103fc2cb8a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:34:02.464925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d184514-c846-4a9e-a188-d4103fc2cb8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:02.466317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d184514-c846-4a9e-a188-d4103fc2cb8a None admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:02.471295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 648/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Tue Aug 9 18:34:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:34:02.477116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7b6e6-2373-4499-8e15-9868eb1c85fd None admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:34:02.576349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7b6e6-2373-4499-8e15-9868eb1c85fd None admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:34:02.608724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9abe25-5f74-4ff9-bc2a-3d96f6c4e380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:34:02.609317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9abe25-5f74-4ff9-bc2a-3d96f6c4e380 None admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:34:02.609619 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9abe25-5f74-4ff9-bc2a-3d96f6c4e380 None admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:34:02.609915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9abe25-5f74-4ff9-bc2a-3d96f6c4e380 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:34:02.610604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9abe25-5f74-4ff9-bc2a-3d96f6c4e380 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:34:02.611945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9abe25-5f74-4ff9-bc2a-3d96f6c4e380 None admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:34:02.669199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 658/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:34:02 2022] POST /identity/v3/endpoints => generated 331 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:34:12.468346 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=30 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:34:12.664543 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:34.388902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a366c7-ca1a-4dfd-93bb-66c1394ffbf5 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:34.389503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a366c7-ca1a-4dfd-93bb-66c1394ffbf5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:34.389503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a366c7-ca1a-4dfd-93bb-66c1394ffbf5 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:34.396066 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d8173-fdfe-44b9-b716-076fd816ce64 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:34.396517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d8173-fdfe-44b9-b716-076fd816ce64 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:34.396606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d8173-fdfe-44b9-b716-076fd816ce64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:34.402318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 649/1307] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:40:34 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:40:34.511212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-304d8173-fdfe-44b9-b716-076fd816ce64 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:40:34.512344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-304d8173-fdfe-44b9-b716-076fd816ce64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:34.666693 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 659/1308] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:40:34.753251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c0610-d782-47b5-a2b8-536b4cfccd77 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:34.753743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c0610-d782-47b5-a2b8-536b4cfccd77 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:34.754352 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c0610-d782-47b5-a2b8-536b4cfccd77 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:34.756764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 650/1309] 10.222.0.57 () {54 vars in 965 bytes} [Tue Aug 9 18:40:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:40:34.766065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd70507-3dea-48c1-b998-034da8a8a656 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:34.766544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd70507-3dea-48c1-b998-034da8a8a656 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:34.766893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd70507-3dea-48c1-b998-034da8a8a656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:34.873845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ecd70507-3dea-48c1-b998-034da8a8a656 None None] MFA Rules not processed for user `88a7cdd610f74330a5da69c0de8472db`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:40:34.875207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ecd70507-3dea-48c1-b998-034da8a8a656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:34.900866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:35.019268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 660/1310] 10.222.0.57 () {58 vars in 1062 bytes} [Tue Aug 9 18:40:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:40:35.029752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c0610-d782-47b5-a2b8-536b4cfccd77 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:40:35.031651 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fe3c0610-d782-47b5-a2b8-536b4cfccd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:35.228069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c0610-d782-47b5-a2b8-536b4cfccd77 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:40:35.331659 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ea202-f726-412b-84f8-fa9846c2a132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:40:35.333081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ea202-f726-412b-84f8-fa9846c2a132 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:35.333612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ea202-f726-412b-84f8-fa9846c2a132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:35.334100 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ea202-f726-412b-84f8-fa9846c2a132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:35.335885 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3b2ea202-f726-412b-84f8-fa9846c2a132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:35.419622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:35.502770 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ea202-f726-412b-84f8-fa9846c2a132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:40:35.504465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ea202-f726-412b-84f8-fa9846c2a132 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:40:35.534336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 651/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:40:37.891090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f053c8-4851-4d00-8386-12ca21706afb None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:37.891645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f053c8-4851-4d00-8386-12ca21706afb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:37.891645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f053c8-4851-4d00-8386-12ca21706afb None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:37.893025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 661/1312] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:40:37.898840 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c1568-c346-4295-95d2-f716d039c1ad None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:37.899380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c1568-c346-4295-95d2-f716d039c1ad None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:37.899380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c1568-c346-4295-95d2-f716d039c1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:37.990251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1d4c1568-c346-4295-95d2-f716d039c1ad None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:40:37.991509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1d4c1568-c346-4295-95d2-f716d039c1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:38.091037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 652/1313] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:40:38.129888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f053c8-4851-4d00-8386-12ca21706afb None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:40:38.131547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b7f053c8-4851-4d00-8386-12ca21706afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:38.245419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f053c8-4851-4d00-8386-12ca21706afb None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:40:38.329724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:38.332132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed0ef3-6326-44cc-95c2-59cbb3626f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:40:38.332745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed0ef3-6326-44cc-95c2-59cbb3626f85 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:38.333023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed0ef3-6326-44cc-95c2-59cbb3626f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:38.333294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed0ef3-6326-44cc-95c2-59cbb3626f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:38.334936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f8ed0ef3-6326-44cc-95c2-59cbb3626f85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:38.457824 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed0ef3-6326-44cc-95c2-59cbb3626f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:40:38.459271 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed0ef3-6326-44cc-95c2-59cbb3626f85 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:40:38.478104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 662/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:40:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:40:41.020266 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb18e2-c602-4dd2-97d1-9ec0f4dcfd41 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:41.020811 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb18e2-c602-4dd2-97d1-9ec0f4dcfd41 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:41.020811 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb18e2-c602-4dd2-97d1-9ec0f4dcfd41 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:41.022200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 653/1315] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:40:41.027473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848de6fd-3f17-4301-a623-070d8ba880d8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:41.027887 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848de6fd-3f17-4301-a623-070d8ba880d8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:41.028020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848de6fd-3f17-4301-a623-070d8ba880d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:41.093725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-848de6fd-3f17-4301-a623-070d8ba880d8 None None] MFA Rules not processed for user `2a5cf324b6904a499ca6308498e53335`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:40:41.094921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-848de6fd-3f17-4301-a623-070d8ba880d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:41.146350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:41.190173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 663/1316] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:40:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:40:41.230268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb18e2-c602-4dd2-97d1-9ec0f4dcfd41 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:40:41.231957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cbdb18e2-c602-4dd2-97d1-9ec0f4dcfd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:41.241806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:41.338609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb18e2-c602-4dd2-97d1-9ec0f4dcfd41 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:40:41.412018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c0b66-13a2-4e87-8426-f734038d8640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:40:41.412621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c0b66-13a2-4e87-8426-f734038d8640 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:41.412901 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c0b66-13a2-4e87-8426-f734038d8640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:41.413174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c0b66-13a2-4e87-8426-f734038d8640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:41.415439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fc5c0b66-13a2-4e87-8426-f734038d8640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:41.477215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:41.528585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c0b66-13a2-4e87-8426-f734038d8640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:40:41.530261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c0b66-13a2-4e87-8426-f734038d8640 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:40:41.555203 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 654/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:40:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:40:44.653915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16112ca8-ef2d-4881-a337-0da1c444f518 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:44.654488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16112ca8-ef2d-4881-a337-0da1c444f518 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:44.654488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16112ca8-ef2d-4881-a337-0da1c444f518 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:44.655800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 664/1318] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:40:44.660830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8ba9a-9350-4653-863d-0d1be751cbea None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:44.661287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8ba9a-9350-4653-863d-0d1be751cbea None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:44.661399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8ba9a-9350-4653-863d-0d1be751cbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:44.726861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8cd8ba9a-9350-4653-863d-0d1be751cbea None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:40:44.728259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8cd8ba9a-9350-4653-863d-0d1be751cbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:44.946021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 655/1319] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:40:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:40:44.997826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5877a-0c33-4b3f-9afb-523f35b5b855 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:44.998198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5877a-0c33-4b3f-9afb-523f35b5b855 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:44.998362 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5877a-0c33-4b3f-9afb-523f35b5b855 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:44.999708 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 665/1320] 10.222.0.57 () {54 vars in 965 bytes} [Tue Aug 9 18:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:40:45.005314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594e9ad-93a5-447e-8bb7-0271d6cc1701 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:45.005634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594e9ad-93a5-447e-8bb7-0271d6cc1701 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:45.005912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594e9ad-93a5-447e-8bb7-0271d6cc1701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:45.027961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:45.082085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e594e9ad-93a5-447e-8bb7-0271d6cc1701 None None] MFA Rules not processed for user `88a7cdd610f74330a5da69c0de8472db`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:40:45.083277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e594e9ad-93a5-447e-8bb7-0271d6cc1701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:45.157178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 656/1321] 10.222.0.57 () {58 vars in 1062 bytes} [Tue Aug 9 18:40:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:40:45.166114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5877a-0c33-4b3f-9afb-523f35b5b855 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:40:45.167640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-adc5877a-0c33-4b3f-9afb-523f35b5b855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:45.271471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5877a-0c33-4b3f-9afb-523f35b5b855 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:40:45.345490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bb053-baf5-4930-a18d-0e7d753e8bbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:40:45.346103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bb053-baf5-4930-a18d-0e7d753e8bbe service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:45.346385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bb053-baf5-4930-a18d-0e7d753e8bbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:45.346657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bb053-baf5-4930-a18d-0e7d753e8bbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:45.348262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a45bb053-baf5-4930-a18d-0e7d753e8bbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:45.355674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:45.458091 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bb053-baf5-4930-a18d-0e7d753e8bbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:40:45.459526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bb053-baf5-4930-a18d-0e7d753e8bbe service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:40:45.478365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 666/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:40:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:40:49.099023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799693d-64c7-4873-b9c1-0936f38d2a25 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:49.099629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799693d-64c7-4873-b9c1-0936f38d2a25 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:49.099629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799693d-64c7-4873-b9c1-0936f38d2a25 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:49.100878 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 657/1323] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:40:49.106219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ac7c4-13ff-4c07-8dd1-654f54bed8da None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:49.106660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ac7c4-13ff-4c07-8dd1-654f54bed8da None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:49.106764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ac7c4-13ff-4c07-8dd1-654f54bed8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:49.174551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-087ac7c4-13ff-4c07-8dd1-654f54bed8da None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:40:49.175760 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-087ac7c4-13ff-4c07-8dd1-654f54bed8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:49.237903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:49.282936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 667/1324] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:40:49.319204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799693d-64c7-4873-b9c1-0936f38d2a25 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:40:49.320726 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1799693d-64c7-4873-b9c1-0936f38d2a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:49.431079 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799693d-64c7-4873-b9c1-0936f38d2a25 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:40:49.444105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:49.507255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf90879-5ac1-4db2-9597-73e326950695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:40:49.507858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf90879-5ac1-4db2-9597-73e326950695 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:49.508134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf90879-5ac1-4db2-9597-73e326950695 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:49.508459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf90879-5ac1-4db2-9597-73e326950695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:49.510598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6bf90879-5ac1-4db2-9597-73e326950695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:49.618819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf90879-5ac1-4db2-9597-73e326950695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:40:49.620249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf90879-5ac1-4db2-9597-73e326950695 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:40:49.638977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 658/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:40:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:40:51.721122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96767e6e-c80f-4fd3-8bd5-bc68675c24d4 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:51.721674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96767e6e-c80f-4fd3-8bd5-bc68675c24d4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:51.721674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96767e6e-c80f-4fd3-8bd5-bc68675c24d4 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:51.723008 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 668/1326] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:40:51.728715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f141f-28b3-4743-a07a-dc342d6787ce None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:51.729146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f141f-28b3-4743-a07a-dc342d6787ce None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:51.729315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f141f-28b3-4743-a07a-dc342d6787ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:51.776749 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:51.795349 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-205f141f-28b3-4743-a07a-dc342d6787ce None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:40:51.796549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-205f141f-28b3-4743-a07a-dc342d6787ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:51.895441 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 659/1327] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:40:51.932205 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96767e6e-c80f-4fd3-8bd5-bc68675c24d4 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:40:51.933800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-96767e6e-c80f-4fd3-8bd5-bc68675c24d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:52.043435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96767e6e-c80f-4fd3-8bd5-bc68675c24d4 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:40:52.121672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b2770-fc3a-4d34-9a9c-62128f0cbe7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:40:52.122298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b2770-fc3a-4d34-9a9c-62128f0cbe7e service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:52.122581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b2770-fc3a-4d34-9a9c-62128f0cbe7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:52.122856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b2770-fc3a-4d34-9a9c-62128f0cbe7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:52.124611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4f9b2770-fc3a-4d34-9a9c-62128f0cbe7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:52.143980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:52.245791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b2770-fc3a-4d34-9a9c-62128f0cbe7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:40:52.247241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b2770-fc3a-4d34-9a9c-62128f0cbe7e service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:40:52.266285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 669/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:40:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:40:54.374422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc3008-9afe-4947-b0da-38ff52dc60e5 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:54.374953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc3008-9afe-4947-b0da-38ff52dc60e5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:54.374953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc3008-9afe-4947-b0da-38ff52dc60e5 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:54.376380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 660/1329] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:40:54.381567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bb55f-5c62-4194-8823-cef7429edabf None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:54.381997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bb55f-5c62-4194-8823-cef7429edabf None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:54.382100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bb55f-5c62-4194-8823-cef7429edabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:54.449709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1e6bb55f-5c62-4194-8823-cef7429edabf None None] MFA Rules not processed for user `2a5cf324b6904a499ca6308498e53335`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:40:54.450810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1e6bb55f-5c62-4194-8823-cef7429edabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:54.523160 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:54.561561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 670/1330] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:40:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:40:54.603108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc3008-9afe-4947-b0da-38ff52dc60e5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:40:54.604761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fefc3008-9afe-4947-b0da-38ff52dc60e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:54.709312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc3008-9afe-4947-b0da-38ff52dc60e5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:40:54.729547 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:54.783707 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2841cc0-d683-4b5c-845f-20424cb88b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:40:54.784322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2841cc0-d683-4b5c-845f-20424cb88b2f service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:54.784598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2841cc0-d683-4b5c-845f-20424cb88b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:54.784867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2841cc0-d683-4b5c-845f-20424cb88b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:54.786529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c2841cc0-d683-4b5c-845f-20424cb88b2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:54.897441 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2841cc0-d683-4b5c-845f-20424cb88b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:40:54.898917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2841cc0-d683-4b5c-845f-20424cb88b2f service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:40:54.917407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 661/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:40:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:40:57.398090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a5fae-e003-4539-882e-e2862541d040 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:57.398598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a5fae-e003-4539-882e-e2862541d040 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:57.398754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a5fae-e003-4539-882e-e2862541d040 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:57.400593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 671/1332] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:40:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:40:57.405893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9037b-f796-4e12-9368-92641347e08a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:57.406283 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9037b-f796-4e12-9368-92641347e08a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:57.406437 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9037b-f796-4e12-9368-92641347e08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:57.470042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:57.472864 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-2ee9037b-f796-4e12-9368-92641347e08a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:40:57.474116 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2ee9037b-f796-4e12-9368-92641347e08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:57.570249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 662/1333] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:40:57.606992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a5fae-e003-4539-882e-e2862541d040 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:40:57.608522 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-632a5fae-e003-4539-882e-e2862541d040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:57.709403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a5fae-e003-4539-882e-e2862541d040 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:40:57.783424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2f8f7-177f-472a-8e7b-9e4f2ecf72ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:40:57.784024 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2f8f7-177f-472a-8e7b-9e4f2ecf72ae service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:57.784304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2f8f7-177f-472a-8e7b-9e4f2ecf72ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:57.784577 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2f8f7-177f-472a-8e7b-9e4f2ecf72ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:57.786216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-15b2f8f7-177f-472a-8e7b-9e4f2ecf72ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:40:57.814039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:40:57.896300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2f8f7-177f-472a-8e7b-9e4f2ecf72ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:40:57.897789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2f8f7-177f-472a-8e7b-9e4f2ecf72ae service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:40:57.917203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 672/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:40:59.938873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb23e14-96de-4d2e-9609-f35dcd02b785 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:59.939419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb23e14-96de-4d2e-9609-f35dcd02b785 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:59.939419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb23e14-96de-4d2e-9609-f35dcd02b785 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:40:59.940836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 663/1335] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:40:59.946033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220697c5-6f60-4ff3-80ae-51a74df0f533 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:40:59.946646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220697c5-6f60-4ff3-80ae-51a74df0f533 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:40:59.946646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220697c5-6f60-4ff3-80ae-51a74df0f533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:00.011742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-220697c5-6f60-4ff3-80ae-51a74df0f533 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:00.012947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-220697c5-6f60-4ff3-80ae-51a74df0f533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:00.087303 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:00.109659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 673/1336] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:00.150103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb23e14-96de-4d2e-9609-f35dcd02b785 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:00.151689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-edb23e14-96de-4d2e-9609-f35dcd02b785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:00.253054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb23e14-96de-4d2e-9609-f35dcd02b785 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:00.282503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:00.327637 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4a382-fa54-4514-a537-18a7935af7d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:00.328235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4a382-fa54-4514-a537-18a7935af7d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:00.328512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4a382-fa54-4514-a537-18a7935af7d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:00.328782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4a382-fa54-4514-a537-18a7935af7d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:00.330398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5ef4a382-fa54-4514-a537-18a7935af7d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:00.436489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4a382-fa54-4514-a537-18a7935af7d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:00.437956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4a382-fa54-4514-a537-18a7935af7d8 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:00.456813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 664/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:03.153545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbccdb7-9591-4794-baf8-603938002fc4 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:03.154089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbccdb7-9591-4794-baf8-603938002fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:03.154233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbccdb7-9591-4794-baf8-603938002fc4 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:03.155497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 674/1338] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:03.160775 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3545b0b-d2ea-4a99-8778-c9af00701ade None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:03.161180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3545b0b-d2ea-4a99-8778-c9af00701ade None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:03.161322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3545b0b-d2ea-4a99-8778-c9af00701ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:03.242984 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c3545b0b-d2ea-4a99-8778-c9af00701ade None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:03.244191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c3545b0b-d2ea-4a99-8778-c9af00701ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:03.253022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:03.341728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 665/1339] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:03.378710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbccdb7-9591-4794-baf8-603938002fc4 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:03.380222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3bbccdb7-9591-4794-baf8-603938002fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:03.483257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbccdb7-9591-4794-baf8-603938002fc4 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:03.559072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4cc7e-8040-4963-a19f-c9f21fe5e97f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:03.559730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4cc7e-8040-4963-a19f-c9f21fe5e97f service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:03.560012 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4cc7e-8040-4963-a19f-c9f21fe5e97f service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:03.560287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4cc7e-8040-4963-a19f-c9f21fe5e97f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:03.561978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-18d4cc7e-8040-4963-a19f-c9f21fe5e97f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:03.598855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:03.676214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4cc7e-8040-4963-a19f-c9f21fe5e97f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:03.677704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4cc7e-8040-4963-a19f-c9f21fe5e97f service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:03.696921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 675/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:07.974491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f9b59-cc90-4a2c-b77c-8d69b3c6e6ff None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:07.975044 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f9b59-cc90-4a2c-b77c-8d69b3c6e6ff None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:07.975044 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f9b59-cc90-4a2c-b77c-8d69b3c6e6ff None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:07.976415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 666/1341] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:07.982056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216fb36-b51f-4b2f-9272-7bf8ddaf10d7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:07.982540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216fb36-b51f-4b2f-9272-7bf8ddaf10d7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:07.983205 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216fb36-b51f-4b2f-9272-7bf8ddaf10d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:08.172477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2216fb36-b51f-4b2f-9272-7bf8ddaf10d7 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:08.173722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2216fb36-b51f-4b2f-9272-7bf8ddaf10d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:08.261958 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:08.277582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 676/1342] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:08.317625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f9b59-cc90-4a2c-b77c-8d69b3c6e6ff None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:08.321081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6e1f9b59-cc90-4a2c-b77c-8d69b3c6e6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:08.428900 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f9b59-cc90-4a2c-b77c-8d69b3c6e6ff None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:08.469339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:08.505151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0315803-2302-4861-ba0c-b6b9ea87738e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:08.505837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0315803-2302-4861-ba0c-b6b9ea87738e service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:08.506314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0315803-2302-4861-ba0c-b6b9ea87738e service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:08.506759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0315803-2302-4861-ba0c-b6b9ea87738e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:08.509268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d0315803-2302-4861-ba0c-b6b9ea87738e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:08.623556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0315803-2302-4861-ba0c-b6b9ea87738e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:08.625010 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0315803-2302-4861-ba0c-b6b9ea87738e service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:08.647714 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 667/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:11.711039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792450e5-2071-4397-9774-8bcb6d36d3bb None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:11.711608 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792450e5-2071-4397-9774-8bcb6d36d3bb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:11.711721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792450e5-2071-4397-9774-8bcb6d36d3bb None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:11.713283 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 677/1344] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:11.718454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef569d8-23f7-4bed-8539-225675b32256 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:11.718919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef569d8-23f7-4bed-8539-225675b32256 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:11.719016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef569d8-23f7-4bed-8539-225675b32256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:11.784870 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0ef569d8-23f7-4bed-8539-225675b32256 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:11.786022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0ef569d8-23f7-4bed-8539-225675b32256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:11.804545 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:11.883301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 668/1345] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:11.920544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792450e5-2071-4397-9774-8bcb6d36d3bb None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:11.922096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-792450e5-2071-4397-9774-8bcb6d36d3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:12.025654 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792450e5-2071-4397-9774-8bcb6d36d3bb None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:12.100816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5d998-c8d4-449f-afe3-52c1989f2368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:12.106480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5d998-c8d4-449f-afe3-52c1989f2368 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:12.106480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5d998-c8d4-449f-afe3-52c1989f2368 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:12.106480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5d998-c8d4-449f-afe3-52c1989f2368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:12.106480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-10d5d998-c8d4-449f-afe3-52c1989f2368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:12.151242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:12.220147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5d998-c8d4-449f-afe3-52c1989f2368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:12.221613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5d998-c8d4-449f-afe3-52c1989f2368 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:12.240879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 678/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:16.566670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86c265-f74f-4339-9960-ce51ee11dbea None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:16.567192 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86c265-f74f-4339-9960-ce51ee11dbea None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:16.567192 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86c265-f74f-4339-9960-ce51ee11dbea None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:16.568588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 669/1347] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:16.573971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eeae2-7091-4b27-9bcd-6d1a98343a81 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:16.574419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eeae2-7091-4b27-9bcd-6d1a98343a81 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:16.574517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eeae2-7091-4b27-9bcd-6d1a98343a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:16.651430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-dc0eeae2-7091-4b27-9bcd-6d1a98343a81 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:16.652661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dc0eeae2-7091-4b27-9bcd-6d1a98343a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:16.761351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:16.763912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 679/1348] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:16.803215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86c265-f74f-4339-9960-ce51ee11dbea None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:16.805114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fa86c265-f74f-4339-9960-ce51ee11dbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:16.925796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86c265-f74f-4339-9960-ce51ee11dbea None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:16.978003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:17.006090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a68dd0-0901-4fb3-9dce-2acadd7de09a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:17.006699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a68dd0-0901-4fb3-9dce-2acadd7de09a service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:17.007005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a68dd0-0901-4fb3-9dce-2acadd7de09a service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:17.007796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a68dd0-0901-4fb3-9dce-2acadd7de09a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:17.010016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-20a68dd0-0901-4fb3-9dce-2acadd7de09a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:17.128440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a68dd0-0901-4fb3-9dce-2acadd7de09a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:17.129920 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a68dd0-0901-4fb3-9dce-2acadd7de09a service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:17.149580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 670/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:22.055762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2a1e2-1f10-4264-bbf6-638e78022dcd None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:22.055762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2a1e2-1f10-4264-bbf6-638e78022dcd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:22.056245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2a1e2-1f10-4264-bbf6-638e78022dcd None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:22.057386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 680/1350] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:41:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:22.063540 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9fb16-1db4-4044-be49-cf440f63a90e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:22.063998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9fb16-1db4-4044-be49-cf440f63a90e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:22.064108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9fb16-1db4-4044-be49-cf440f63a90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:22.159256 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-dcc9fb16-1db4-4044-be49-cf440f63a90e None None] MFA Rules not processed for user `5e0aa6b215f646c280af245a89ccf849`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:22.160665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dcc9fb16-1db4-4044-be49-cf440f63a90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:22.196591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:22.298587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 671/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:41:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:26.446742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48961b-64a8-4220-969e-c11b70f7931e None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:26.447272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48961b-64a8-4220-969e-c11b70f7931e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:26.447272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48961b-64a8-4220-969e-c11b70f7931e None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:26.448574 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 681/1352] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:26.455133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9e47b-b842-4201-92aa-018043fed110 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:26.455549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9e47b-b842-4201-92aa-018043fed110 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:26.455770 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9e47b-b842-4201-92aa-018043fed110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:26.536445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d9e9e47b-b842-4201-92aa-018043fed110 None None] MFA Rules not processed for user `5e0aa6b215f646c280af245a89ccf849`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:26.537614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d9e9e47b-b842-4201-92aa-018043fed110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:26.612610 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:26.652519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 672/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:41:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:28.235206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504ea5-da4f-4b43-946e-9a76e33760af None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:28.235785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504ea5-da4f-4b43-946e-9a76e33760af None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:28.235785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504ea5-da4f-4b43-946e-9a76e33760af None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:28.237301 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 682/1354] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:28.243042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d061d-bb54-4840-8205-99fd98fcb223 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:28.243607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d061d-bb54-4840-8205-99fd98fcb223 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:28.243607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d061d-bb54-4840-8205-99fd98fcb223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:28.331819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5b0d061d-bb54-4840-8205-99fd98fcb223 None None] MFA Rules not processed for user `5e0aa6b215f646c280af245a89ccf849`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:28.333055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5b0d061d-bb54-4840-8205-99fd98fcb223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:28.446459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 673/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:41:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:41:31.758850 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c02a6e-ad30-45d8-aacb-2ced92d706d8 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:31.759422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c02a6e-ad30-45d8-aacb-2ced92d706d8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:31.759422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c02a6e-ad30-45d8-aacb-2ced92d706d8 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:31.760703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 683/1356] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:31.765858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a91e6-aad3-4745-8987-57e85e30a9f3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:31.766261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a91e6-aad3-4745-8987-57e85e30a9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:31.766408 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a91e6-aad3-4745-8987-57e85e30a9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:31.823080 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:31.863433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-119a91e6-aad3-4745-8987-57e85e30a9f3 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:31.864631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-119a91e6-aad3-4745-8987-57e85e30a9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:31.976991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 674/1357] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:32.033045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6074b3c-dfdb-46b2-bafb-08e90547317e None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:32.033326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6074b3c-dfdb-46b2-bafb-08e90547317e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:32.033619 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6074b3c-dfdb-46b2-bafb-08e90547317e None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:32.035418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 684/1358] 10.222.0.57 () {54 vars in 962 bytes} [Tue Aug 9 18:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:32.043074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0f6e2-66fc-4c8d-811a-3b389676fde1 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:32.043285 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0f6e2-66fc-4c8d-811a-3b389676fde1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:32.043520 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0f6e2-66fc-4c8d-811a-3b389676fde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:32.127858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-bae0f6e2-66fc-4c8d-811a-3b389676fde1 None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:32.129189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bae0f6e2-66fc-4c8d-811a-3b389676fde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:32.187754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:32.272955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 675/1359] 10.222.0.57 () {58 vars in 1059 bytes} [Tue Aug 9 18:41:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:32.282068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6074b3c-dfdb-46b2-bafb-08e90547317e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:32.283892 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f6074b3c-dfdb-46b2-bafb-08e90547317e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:32.460408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6074b3c-dfdb-46b2-bafb-08e90547317e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:32.536435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:32.582482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea6f04-1474-46c0-8e8d-3858f4f646e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:32.583096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea6f04-1474-46c0-8e8d-3858f4f646e0 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:32.583368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea6f04-1474-46c0-8e8d-3858f4f646e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:32.583905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea6f04-1474-46c0-8e8d-3858f4f646e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:32.585554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f9ea6f04-1474-46c0-8e8d-3858f4f646e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:32.685085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea6f04-1474-46c0-8e8d-3858f4f646e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:32.686884 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea6f04-1474-46c0-8e8d-3858f4f646e0 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:32.706501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 685/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:35.044684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffac51-29f8-43b9-8aad-d919e64f9400 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:35.045280 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffac51-29f8-43b9-8aad-d919e64f9400 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:35.045564 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffac51-29f8-43b9-8aad-d919e64f9400 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:35.047094 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 676/1361] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:35.052625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5363e2-a5fa-4cac-a9a9-459c1833c9a7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:35.053133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5363e2-a5fa-4cac-a9a9-459c1833c9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:35.053226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5363e2-a5fa-4cac-a9a9-459c1833c9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:35.142098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0e5363e2-a5fa-4cac-a9a9-459c1833c9a7 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:35.143447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0e5363e2-a5fa-4cac-a9a9-459c1833c9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:35.190108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:35.253265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 686/1362] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:35.298990 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffac51-29f8-43b9-8aad-d919e64f9400 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:35.302853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ecffac51-29f8-43b9-8aad-d919e64f9400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:35.481728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffac51-29f8-43b9-8aad-d919e64f9400 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:35.492439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:35.635907 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da3940-51fc-42ad-98fc-405b1900d407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:35.636532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da3940-51fc-42ad-98fc-405b1900d407 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:35.636811 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da3940-51fc-42ad-98fc-405b1900d407 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:35.637083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da3940-51fc-42ad-98fc-405b1900d407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:35.638697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-57da3940-51fc-42ad-98fc-405b1900d407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:35.735903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da3940-51fc-42ad-98fc-405b1900d407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:35.737562 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da3940-51fc-42ad-98fc-405b1900d407 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:35.757170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 677/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:38.237267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8e908-5daf-4a56-ad0f-0f37c8e00c18 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:38.238297 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8e908-5daf-4a56-ad0f-0f37c8e00c18 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:38.238578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8e908-5daf-4a56-ad0f-0f37c8e00c18 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:38.239934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 687/1364] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:38.246018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b267c8-2cf6-41a6-be28-f82edcc2d35f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:38.246471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b267c8-2cf6-41a6-be28-f82edcc2d35f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:38.246561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b267c8-2cf6-41a6-be28-f82edcc2d35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:38.267805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:38.328400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c7b267c8-2cf6-41a6-be28-f82edcc2d35f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:38.329648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c7b267c8-2cf6-41a6-be28-f82edcc2d35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:38.432779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 678/1365] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:38.487428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e002efa-3ad5-4e19-bef7-631c7a69d670 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:38.487674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e002efa-3ad5-4e19-bef7-631c7a69d670 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:38.490352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e002efa-3ad5-4e19-bef7-631c7a69d670 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:38.490352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 688/1366] 10.222.0.57 () {54 vars in 962 bytes} [Tue Aug 9 18:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:38.496715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36135757-b2dc-4876-8cce-4fe8f57ded32 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:38.497007 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36135757-b2dc-4876-8cce-4fe8f57ded32 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:38.497278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36135757-b2dc-4876-8cce-4fe8f57ded32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:38.582067 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-36135757-b2dc-4876-8cce-4fe8f57ded32 None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:38.583273 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-36135757-b2dc-4876-8cce-4fe8f57ded32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:38.620567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:38.723972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 679/1367] 10.222.0.57 () {58 vars in 1059 bytes} [Tue Aug 9 18:41:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:38.732029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e002efa-3ad5-4e19-bef7-631c7a69d670 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:38.733787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7e002efa-3ad5-4e19-bef7-631c7a69d670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:38.894008 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e002efa-3ad5-4e19-bef7-631c7a69d670 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:38.917661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:39.015726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6980b-8101-4eb0-829a-7c051209e7c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:39.016486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6980b-8101-4eb0-829a-7c051209e7c0 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:39.016736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6980b-8101-4eb0-829a-7c051209e7c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:39.017007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6980b-8101-4eb0-829a-7c051209e7c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:39.018645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d2e6980b-8101-4eb0-829a-7c051209e7c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:39.126348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6980b-8101-4eb0-829a-7c051209e7c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:39.128024 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6980b-8101-4eb0-829a-7c051209e7c0 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:39.156844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 689/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:40.142256 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c630-8c51-4295-897e-bbd03bdd1ad6 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:40.142816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c630-8c51-4295-897e-bbd03bdd1ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:40.142816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c630-8c51-4295-897e-bbd03bdd1ad6 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:40.144390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 680/1369] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:40.149420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada368f2-2734-4f54-9b4b-13522eddc4ff None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:40.149870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada368f2-2734-4f54-9b4b-13522eddc4ff None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:40.150075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada368f2-2734-4f54-9b4b-13522eddc4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:40.184197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:40.219550 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ada368f2-2734-4f54-9b4b-13522eddc4ff None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:40.220764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ada368f2-2734-4f54-9b4b-13522eddc4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:40.331838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 690/1370] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:40.711563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0afbf2-153b-4169-bf74-49278bbd9f15 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:40.712067 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0afbf2-153b-4169-bf74-49278bbd9f15 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:40.712503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0afbf2-153b-4169-bf74-49278bbd9f15 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:40.713990 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 681/1371] 10.222.0.57 () {54 vars in 964 bytes} [Tue Aug 9 18:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:40.718982 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af82478-1675-465f-a17a-7fcfa12224a5 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:40.719445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af82478-1675-465f-a17a-7fcfa12224a5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:40.719865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af82478-1675-465f-a17a-7fcfa12224a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:40.798129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0af82478-1675-465f-a17a-7fcfa12224a5 None None] MFA Rules not processed for user `77300e25b9004a598f8f7f12f3cf4866`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:40.799320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0af82478-1675-465f-a17a-7fcfa12224a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:40.848045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:40.879402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 691/1372] 10.222.0.57 () {58 vars in 1061 bytes} [Tue Aug 9 18:41:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:40.886818 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0afbf2-153b-4169-bf74-49278bbd9f15 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:40.888730 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4c0afbf2-153b-4169-bf74-49278bbd9f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:41.003665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0afbf2-153b-4169-bf74-49278bbd9f15 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:41.045830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:41.082777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9ebf7-507a-4bb9-a957-8f1600e42f0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:41.083388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9ebf7-507a-4bb9-a957-8f1600e42f0e service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:41.083667 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9ebf7-507a-4bb9-a957-8f1600e42f0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:41.083937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9ebf7-507a-4bb9-a957-8f1600e42f0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:41.086150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-41a9ebf7-507a-4bb9-a957-8f1600e42f0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:41.212402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9ebf7-507a-4bb9-a957-8f1600e42f0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:41.213867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9ebf7-507a-4bb9-a957-8f1600e42f0e service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:41.233686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 682/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:41.431077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b87b05-5471-4778-ab36-6a2029b5c3c9 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:41.431749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b87b05-5471-4778-ab36-6a2029b5c3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:41.431749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b87b05-5471-4778-ab36-6a2029b5c3c9 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:41.436853 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 692/1374] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:41.438655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0275b-076b-4fc0-a694-eeda5c3daf9f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:41.438655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0275b-076b-4fc0-a694-eeda5c3daf9f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:41.439420 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0275b-076b-4fc0-a694-eeda5c3daf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:41.533616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a8c0275b-076b-4fc0-a694-eeda5c3daf9f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:41.535567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a8c0275b-076b-4fc0-a694-eeda5c3daf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:41.565605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:41.655593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 683/1375] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:41:41.701226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b87b05-5471-4778-ab36-6a2029b5c3c9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:41.702901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-27b87b05-5471-4778-ab36-6a2029b5c3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:41.850652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b87b05-5471-4778-ab36-6a2029b5c3c9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:41.902829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:41.967661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ec10b-ae2a-4952-ac8c-4e4424ddc6bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:41.968270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ec10b-ae2a-4952-ac8c-4e4424ddc6bb service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:41.968550 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ec10b-ae2a-4952-ac8c-4e4424ddc6bb service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:41.968825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ec10b-ae2a-4952-ac8c-4e4424ddc6bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:41.970508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-686ec10b-ae2a-4952-ac8c-4e4424ddc6bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:42.063400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ec10b-ae2a-4952-ac8c-4e4424ddc6bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:42.065031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ec10b-ae2a-4952-ac8c-4e4424ddc6bb service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:42.083022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 693/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:44.304754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac29fd0-0e33-404f-aafe-e4029c070a3e None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:44.305693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac29fd0-0e33-404f-aafe-e4029c070a3e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:44.305901 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac29fd0-0e33-404f-aafe-e4029c070a3e None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:44.307501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 684/1377] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:44.313246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0214e22-b027-4d17-a056-c46a3ce65157 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:44.313747 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0214e22-b027-4d17-a056-c46a3ce65157 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:44.313847 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0214e22-b027-4d17-a056-c46a3ce65157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:44.384693 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a0214e22-b027-4d17-a056-c46a3ce65157 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:44.385971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a0214e22-b027-4d17-a056-c46a3ce65157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:44.392903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:44.519712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 694/1378] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:44.564192 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac29fd0-0e33-404f-aafe-e4029c070a3e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:44.565744 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-aac29fd0-0e33-404f-aafe-e4029c070a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:44.720415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac29fd0-0e33-404f-aafe-e4029c070a3e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:44.730019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:44.851197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fa6d3-2158-4f74-9fde-93858f433c53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:44.851818 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fa6d3-2158-4f74-9fde-93858f433c53 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:44.852155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fa6d3-2158-4f74-9fde-93858f433c53 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:44.852419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fa6d3-2158-4f74-9fde-93858f433c53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:44.854303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3e6fa6d3-2158-4f74-9fde-93858f433c53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:44.982510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fa6d3-2158-4f74-9fde-93858f433c53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:44.984479 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fa6d3-2158-4f74-9fde-93858f433c53 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:45.010258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 685/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:47.221157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f1e1-e77d-4253-99c9-2a6a448cdb43 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:47.222783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f1e1-e77d-4253-99c9-2a6a448cdb43 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:47.222783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f1e1-e77d-4253-99c9-2a6a448cdb43 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:47.227363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 695/1380] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:47.229077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173a806-7599-4e99-8d33-45e1ddda9151 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:47.229853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173a806-7599-4e99-8d33-45e1ddda9151 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:47.229853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173a806-7599-4e99-8d33-45e1ddda9151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:47.241931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:47.312174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d173a806-7599-4e99-8d33-45e1ddda9151 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:47.314070 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d173a806-7599-4e99-8d33-45e1ddda9151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:47.435440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 686/1381] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:47.481320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f1e1-e77d-4253-99c9-2a6a448cdb43 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:47.483149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ce22f1e1-e77d-4253-99c9-2a6a448cdb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:47.638345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:47.651255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f1e1-e77d-4253-99c9-2a6a448cdb43 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:47.785503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce52155-c9fd-4d43-a9ae-5bf99c4933e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:47.786284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce52155-c9fd-4d43-a9ae-5bf99c4933e5 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:47.786690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce52155-c9fd-4d43-a9ae-5bf99c4933e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:47.787066 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce52155-c9fd-4d43-a9ae-5bf99c4933e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:47.789034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2ce52155-c9fd-4d43-a9ae-5bf99c4933e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:47.907406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce52155-c9fd-4d43-a9ae-5bf99c4933e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:47.909320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce52155-c9fd-4d43-a9ae-5bf99c4933e5 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:47.922114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:47.932535 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 696/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:41:50.200806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a6471-7f49-4e5b-a542-c22a7dcbb7e4 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:50.201355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a6471-7f49-4e5b-a542-c22a7dcbb7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:50.201355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a6471-7f49-4e5b-a542-c22a7dcbb7e4 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:50.202785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 687/1383] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:50.208161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce1d96-78b7-4e97-aa36-2946481d52e6 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:50.208632 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce1d96-78b7-4e97-aa36-2946481d52e6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:50.208741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce1d96-78b7-4e97-aa36-2946481d52e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:50.280085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-42ce1d96-78b7-4e97-aa36-2946481d52e6 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:50.281405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-42ce1d96-78b7-4e97-aa36-2946481d52e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:50.401633 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 697/1384] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:50.445681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a6471-7f49-4e5b-a542-c22a7dcbb7e4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:50.447824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-369a6471-7f49-4e5b-a542-c22a7dcbb7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:50.542247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:50.611615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a6471-7f49-4e5b-a542-c22a7dcbb7e4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:50.733374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0990a1-8c45-407b-ad1e-a3d2817397c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:50.734026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0990a1-8c45-407b-ad1e-a3d2817397c6 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:50.734317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0990a1-8c45-407b-ad1e-a3d2817397c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:50.734591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0990a1-8c45-407b-ad1e-a3d2817397c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:50.736254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ce0990a1-8c45-407b-ad1e-a3d2817397c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:50.797130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:50.838958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0990a1-8c45-407b-ad1e-a3d2817397c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:50.840567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0990a1-8c45-407b-ad1e-a3d2817397c6 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:50.859844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 688/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:53.016287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f84d6-06d4-4aad-b00b-7f9dff65b10f None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:53.016851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f84d6-06d4-4aad-b00b-7f9dff65b10f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:53.016951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f84d6-06d4-4aad-b00b-7f9dff65b10f None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:53.018828 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 698/1386] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:41:53.026214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0ca8f-ceef-49d9-8b22-b8c663be0ec7 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:53.026715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0ca8f-ceef-49d9-8b22-b8c663be0ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:53.026942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0ca8f-ceef-49d9-8b22-b8c663be0ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:53.115996 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b6d0ca8f-ceef-49d9-8b22-b8c663be0ec7 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:53.117178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b6d0ca8f-ceef-49d9-8b22-b8c663be0ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:53.229089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 689/1387] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:53.286077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f84d6-06d4-4aad-b00b-7f9dff65b10f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:53.288718 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9a1f84d6-06d4-4aad-b00b-7f9dff65b10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:53.386382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:53.514584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f84d6-06d4-4aad-b00b-7f9dff65b10f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:53.690884 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb60d7d-2bc9-4ef7-89d5-912ab7582c74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:53.692068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb60d7d-2bc9-4ef7-89d5-912ab7582c74 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:53.692500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb60d7d-2bc9-4ef7-89d5-912ab7582c74 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:53.692964 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb60d7d-2bc9-4ef7-89d5-912ab7582c74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:53.695535 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1bb60d7d-2bc9-4ef7-89d5-912ab7582c74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:53.746011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:53.814728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb60d7d-2bc9-4ef7-89d5-912ab7582c74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:53.816335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb60d7d-2bc9-4ef7-89d5-912ab7582c74 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:53.843130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 699/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:56.194942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd82cb-dbba-43ea-92be-7eb167a9bc56 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:56.195546 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd82cb-dbba-43ea-92be-7eb167a9bc56 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:56.195546 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd82cb-dbba-43ea-92be-7eb167a9bc56 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:56.196909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 690/1389] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:56.202661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7c65e-9f91-490b-91f1-2120383b1e60 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:56.203115 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7c65e-9f91-490b-91f1-2120383b1e60 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:56.203231 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7c65e-9f91-490b-91f1-2120383b1e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:56.274271 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4ea7c65e-9f91-490b-91f1-2120383b1e60 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:56.275513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4ea7c65e-9f91-490b-91f1-2120383b1e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:56.379414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 700/1390] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:56.424568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd82cb-dbba-43ea-92be-7eb167a9bc56 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:56.426094 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d0bd82cb-dbba-43ea-92be-7eb167a9bc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:56.459153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:56.583171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd82cb-dbba-43ea-92be-7eb167a9bc56 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:56.691496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:56.703561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105fff2-977f-4bb0-86fa-c02290aa5194 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:56.704175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105fff2-977f-4bb0-86fa-c02290aa5194 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:56.704447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105fff2-977f-4bb0-86fa-c02290aa5194 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:56.704720 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105fff2-977f-4bb0-86fa-c02290aa5194 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:56.706361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c105fff2-977f-4bb0-86fa-c02290aa5194 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:56.754532 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89198864-46c7-44f4-9c2c-96a83dca166a None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:56.754738 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89198864-46c7-44f4-9c2c-96a83dca166a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:56.755000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89198864-46c7-44f4-9c2c-96a83dca166a None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:56.756374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 701/1391] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:41:56.762372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf35386-05a0-43c0-840c-807a2a95796c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:56.762959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf35386-05a0-43c0-840c-807a2a95796c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:56.763245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf35386-05a0-43c0-840c-807a2a95796c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:56.779184 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:56.819146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105fff2-977f-4bb0-86fa-c02290aa5194 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:56.820808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105fff2-977f-4bb0-86fa-c02290aa5194 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:56.836701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6cf35386-05a0-43c0-840c-807a2a95796c None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:56.837922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6cf35386-05a0-43c0-840c-807a2a95796c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:56.855326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 691/1392] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:56.941907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 702/1393] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:41:56.998651 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a91c8a-87a8-4897-b30e-69e4e301ad28 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:56.998940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a91c8a-87a8-4897-b30e-69e4e301ad28 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:56.999262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a91c8a-87a8-4897-b30e-69e4e301ad28 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:57.000598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 692/1394] 10.222.0.57 () {54 vars in 967 bytes} [Tue Aug 9 18:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:57.005726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32636d-4c8a-42d6-97a2-c8b6ce394ad9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:57.005953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32636d-4c8a-42d6-97a2-c8b6ce394ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:57.006225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32636d-4c8a-42d6-97a2-c8b6ce394ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:57.091615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bb32636d-4c8a-42d6-97a2-c8b6ce394ad9 None None] MFA Rules not processed for user `5a58f9101f2d4785b96f0cc55be4330b`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:57.092869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bb32636d-4c8a-42d6-97a2-c8b6ce394ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:57.139148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:57.195054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 703/1395] 10.222.0.57 () {58 vars in 1064 bytes} [Tue Aug 9 18:41:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:57.202696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a91c8a-87a8-4897-b30e-69e4e301ad28 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:57.204269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-17a91c8a-87a8-4897-b30e-69e4e301ad28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:57.316033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a91c8a-87a8-4897-b30e-69e4e301ad28 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:57.330204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:57.391723 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37565da4-aed8-4df3-8b2b-9b5986aee7b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a58f9101f2d4785b96f0cc55be4330b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:57.392322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37565da4-aed8-4df3-8b2b-9b5986aee7b5 service glance] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:57.392600 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37565da4-aed8-4df3-8b2b-9b5986aee7b5 service glance] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:57.392866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37565da4-aed8-4df3-8b2b-9b5986aee7b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:57.394477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-37565da4-aed8-4df3-8b2b-9b5986aee7b5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:57.506965 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37565da4-aed8-4df3-8b2b-9b5986aee7b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:57.508394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37565da4-aed8-4df3-8b2b-9b5986aee7b5 service glance] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:57.526452 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 693/1396] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:41:58.510275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8c72a-eef3-40c5-8ad2-d50e590456dc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:58.510723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8c72a-eef3-40c5-8ad2-d50e590456dc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:58.510811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8c72a-eef3-40c5-8ad2-d50e590456dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:58.579515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a8c8c72a-eef3-40c5-8ad2-d50e590456dc None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:58.580725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a8c8c72a-eef3-40c5-8ad2-d50e590456dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:58.673338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 704/1397] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:41:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:58.718299 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaab707-3007-4058-99fe-168fecf42871 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:58.718828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaab707-3007-4058-99fe-168fecf42871 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:58.718921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaab707-3007-4058-99fe-168fecf42871 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:58.720390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 694/1398] 10.222.0.57 () {54 vars in 943 bytes} [Tue Aug 9 18:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:41:58.727926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23228e5-7f97-4246-8e62-3c618401703e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:58.728110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23228e5-7f97-4246-8e62-3c618401703e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:58.728478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23228e5-7f97-4246-8e62-3c618401703e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:58.792437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:58.811655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d23228e5-7f97-4246-8e62-3c618401703e None None] MFA Rules not processed for user `2ff735ecfcdf49089b5cfeafc56048cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:58.812919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d23228e5-7f97-4246-8e62-3c618401703e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:58.944257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 705/1399] 10.222.0.57 () {58 vars in 1040 bytes} [Tue Aug 9 18:41:58 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:41:58.953785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaab707-3007-4058-99fe-168fecf42871 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:58.955483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bdaab707-3007-4058-99fe-168fecf42871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:59.001743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:59.043583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47851068-2093-48cf-9b02-b37b40053f3d None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:59.043877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47851068-2093-48cf-9b02-b37b40053f3d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:59.044230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47851068-2093-48cf-9b02-b37b40053f3d None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:59.045680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 706/1400] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:41:59.052198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d74022-0a94-4254-82d7-a16773362690 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:59.052479 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d74022-0a94-4254-82d7-a16773362690 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:59.052749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d74022-0a94-4254-82d7-a16773362690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:59.094468 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaab707-3007-4058-99fe-168fecf42871 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:59.122478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c9d74022-0a94-4254-82d7-a16773362690 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:41:59.123711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c9d74022-0a94-4254-82d7-a16773362690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:59.188399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:59.205641 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2af55b-7490-4fbf-ab86-0a8799d9ec93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:59.206248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2af55b-7490-4fbf-ab86-0a8799d9ec93 service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:59.206521 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2af55b-7490-4fbf-ab86-0a8799d9ec93 service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:59.206791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2af55b-7490-4fbf-ab86-0a8799d9ec93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:59.208432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5c2af55b-7490-4fbf-ab86-0a8799d9ec93 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:59.232490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 707/1401] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:41:59.239414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:59.278950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d74022-0a94-4254-82d7-a16773362690 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:41:59.280165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c9d74022-0a94-4254-82d7-a16773362690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:59.300686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2af55b-7490-4fbf-ab86-0a8799d9ec93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:59.302517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2af55b-7490-4fbf-ab86-0a8799d9ec93 service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:59.314046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 695/1402] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:41:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:41:59.404106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d74022-0a94-4254-82d7-a16773362690 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:41:59.473834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:41:59.521293 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3364ec-c852-472e-a37c-75ce1c748fdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:41:59.521928 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3364ec-c852-472e-a37c-75ce1c748fdb service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:41:59.522762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3364ec-c852-472e-a37c-75ce1c748fdb service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:41:59.523034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3364ec-c852-472e-a37c-75ce1c748fdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:41:59.524638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7e3364ec-c852-472e-a37c-75ce1c748fdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:41:59.616166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3364ec-c852-472e-a37c-75ce1c748fdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:41:59.618015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3364ec-c852-472e-a37c-75ce1c748fdb service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:41:59.639714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 708/1403] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:41:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:42:01.874610 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f4750-ce9e-4dc7-9f38-0b2e65cd122f None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:01.875171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f4750-ce9e-4dc7-9f38-0b2e65cd122f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:01.875171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f4750-ce9e-4dc7-9f38-0b2e65cd122f None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:01.876497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 696/1404] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:01.884092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b98b2-6179-46fd-bf31-cf91315e221d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:01.886903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b98b2-6179-46fd-bf31-cf91315e221d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:01.887473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b98b2-6179-46fd-bf31-cf91315e221d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:01.964091 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4f2b98b2-6179-46fd-bf31-cf91315e221d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:01.965365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4f2b98b2-6179-46fd-bf31-cf91315e221d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:02.004119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:02.078441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 709/1405] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:02.125432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f4750-ce9e-4dc7-9f38-0b2e65cd122f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:02.127291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-939f4750-ce9e-4dc7-9f38-0b2e65cd122f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:02.300355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f4750-ce9e-4dc7-9f38-0b2e65cd122f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:02.325096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:02.447220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781cdc0-572e-4bed-96b4-bbfe9d5d2773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:02.447830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781cdc0-572e-4bed-96b4-bbfe9d5d2773 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:02.448107 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781cdc0-572e-4bed-96b4-bbfe9d5d2773 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:02.448468 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781cdc0-572e-4bed-96b4-bbfe9d5d2773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:02.450384 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1781cdc0-572e-4bed-96b4-bbfe9d5d2773 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:02.562710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781cdc0-572e-4bed-96b4-bbfe9d5d2773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:02.564614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781cdc0-572e-4bed-96b4-bbfe9d5d2773 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:02.587225 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 697/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:04.872821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f393147-735b-4347-9e2c-5d32b597b0e4 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:04.876307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f393147-735b-4347-9e2c-5d32b597b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:04.876584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f393147-735b-4347-9e2c-5d32b597b0e4 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:04.878093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 710/1407] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:42:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:04.883972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1c924-d120-4bcf-b97b-346512f70c0f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:04.884453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1c924-d120-4bcf-b97b-346512f70c0f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:04.884548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1c924-d120-4bcf-b97b-346512f70c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:04.931052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:04.966413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-23c1c924-d120-4bcf-b97b-346512f70c0f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:04.967729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-23c1c924-d120-4bcf-b97b-346512f70c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:05.079642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 698/1408] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:05.135204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f393147-735b-4347-9e2c-5d32b597b0e4 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:05.136250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9f393147-735b-4347-9e2c-5d32b597b0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:05.330492 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f393147-735b-4347-9e2c-5d32b597b0e4 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:05.353062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:05.470090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4cbe0b-7446-4bfa-8cb5-601b40cfac1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:05.470731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cbe0b-7446-4bfa-8cb5-601b40cfac1b service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:05.471075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cbe0b-7446-4bfa-8cb5-601b40cfac1b service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:05.471441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cbe0b-7446-4bfa-8cb5-601b40cfac1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:05.473153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7f4cbe0b-7446-4bfa-8cb5-601b40cfac1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:05.588113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4cbe0b-7446-4bfa-8cb5-601b40cfac1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:05.589770 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4cbe0b-7446-4bfa-8cb5-601b40cfac1b service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:05.609596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 711/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:15.124964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=53 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:15.606916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:23.161977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034b01d-4c47-4cde-affa-72d2ed98662f None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:23.163079 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034b01d-4c47-4cde-affa-72d2ed98662f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:23.163079 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034b01d-4c47-4cde-affa-72d2ed98662f None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:23.164358 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 699/1410] 10.222.0.57 () {54 vars in 918 bytes} [Tue Aug 9 18:42:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:23.169608 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51221f7-97e7-4a6f-8108-5119d6158db2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:23.170048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51221f7-97e7-4a6f-8108-5119d6158db2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:23.170171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51221f7-97e7-4a6f-8108-5119d6158db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:23.249249 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a51221f7-97e7-4a6f-8108-5119d6158db2 None None] MFA Rules not processed for user `5e0aa6b215f646c280af245a89ccf849`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:23.250500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a51221f7-97e7-4a6f-8108-5119d6158db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:23.359816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 712/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Tue Aug 9 18:42:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:23.375119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3533333-7095-4cca-ba01-53f883b5c556 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:23.375403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3533333-7095-4cca-ba01-53f883b5c556 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:23.375674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3533333-7095-4cca-ba01-53f883b5c556 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:23.376982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 700/1412] 10.222.0.57 () {54 vars in 918 bytes} [Tue Aug 9 18:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:23.381789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f261e8-9e68-4d8d-ae19-3e5d6dd412eb None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:23.382071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f261e8-9e68-4d8d-ae19-3e5d6dd412eb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:23.382342 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f261e8-9e68-4d8d-ae19-3e5d6dd412eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:23.461339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:23.463792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a2f261e8-9e68-4d8d-ae19-3e5d6dd412eb None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:23.464999 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a2f261e8-9e68-4d8d-ae19-3e5d6dd412eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:23.582906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 713/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Tue Aug 9 18:42:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:26.117808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be090c1-76f3-4323-a228-c669073224b6 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:26.118395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be090c1-76f3-4323-a228-c669073224b6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:26.118395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be090c1-76f3-4323-a228-c669073224b6 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:26.119681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:26.125054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51d3dc-74e5-40cb-a4c1-270e06145f34 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:26.125506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51d3dc-74e5-40cb-a4c1-270e06145f34 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:26.125647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51d3dc-74e5-40cb-a4c1-270e06145f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:26.192749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1e51d3dc-74e5-40cb-a4c1-270e06145f34 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:26.193980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1e51d3dc-74e5-40cb-a4c1-270e06145f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:26.250137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:26.293381 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:26.337233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be090c1-76f3-4323-a228-c669073224b6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:26.338805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0be090c1-76f3-4323-a228-c669073224b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:26.490583 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be090c1-76f3-4323-a228-c669073224b6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:26.564104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:26.604429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287ffdd-b06f-4e49-85ad-ca2854e59690 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:26.605223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287ffdd-b06f-4e49-85ad-ca2854e59690 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:26.605748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287ffdd-b06f-4e49-85ad-ca2854e59690 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:26.606080 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287ffdd-b06f-4e49-85ad-ca2854e59690 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:26.608026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d287ffdd-b06f-4e49-85ad-ca2854e59690 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:26.696781 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287ffdd-b06f-4e49-85ad-ca2854e59690 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:26.699309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287ffdd-b06f-4e49-85ad-ca2854e59690 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:26.719104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 702/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:27.618534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c3790-437d-4d4c-b13c-602319703d49 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:27.619084 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c3790-437d-4d4c-b13c-602319703d49 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:27.619180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c3790-437d-4d4c-b13c-602319703d49 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:27.620541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:27.625570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a12bb6-2cbc-4a13-94b7-f2cda9ce69d5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:27.625963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a12bb6-2cbc-4a13-94b7-f2cda9ce69d5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:27.626118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a12bb6-2cbc-4a13-94b7-f2cda9ce69d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:27.703803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-10a12bb6-2cbc-4a13-94b7-f2cda9ce69d5 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:27.705002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-10a12bb6-2cbc-4a13-94b7-f2cda9ce69d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:27.733740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:27.804405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:27.957820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c3790-437d-4d4c-b13c-602319703d49 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:27.959374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bb7c3790-437d-4d4c-b13c-602319703d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:28.120604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c3790-437d-4d4c-b13c-602319703d49 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:28.160048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:28.234839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac2496-2e4e-4cca-9cca-3f5532b66673 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:28.235328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac2496-2e4e-4cca-9cca-3f5532b66673 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:28.235661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac2496-2e4e-4cca-9cca-3f5532b66673 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:28.235931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac2496-2e4e-4cca-9cca-3f5532b66673 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:28.237583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-57ac2496-2e4e-4cca-9cca-3f5532b66673 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:28.328382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac2496-2e4e-4cca-9cca-3f5532b66673 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:28.330036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac2496-2e4e-4cca-9cca-3f5532b66673 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:28.350881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 716/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:29.236833 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a8cc5-e548-4afd-be2a-a93ddbe231d0 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:29.237418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a8cc5-e548-4afd-be2a-a93ddbe231d0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:29.237418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a8cc5-e548-4afd-be2a-a93ddbe231d0 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:29.238782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:29.244360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea79f55-8651-431b-ab4a-39360e682e0e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:29.244798 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea79f55-8651-431b-ab4a-39360e682e0e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:29.244937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea79f55-8651-431b-ab4a-39360e682e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:29.300597 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:29.322359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4ea79f55-8651-431b-ab4a-39360e682e0e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:29.323579 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4ea79f55-8651-431b-ab4a-39360e682e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:29.420761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:29.577443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a8cc5-e548-4afd-be2a-a93ddbe231d0 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:29.579012 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-911a8cc5-e548-4afd-be2a-a93ddbe231d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:29.730268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a8cc5-e548-4afd-be2a-a93ddbe231d0 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:29.742994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:29.846370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e454e9-e351-4f0a-881b-505568f577ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:29.846879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e454e9-e351-4f0a-881b-505568f577ad service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:29.847153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e454e9-e351-4f0a-881b-505568f577ad service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:29.847422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e454e9-e351-4f0a-881b-505568f577ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:29.849003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-00e454e9-e351-4f0a-881b-505568f577ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:29.937315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e454e9-e351-4f0a-881b-505568f577ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:29.938953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e454e9-e351-4f0a-881b-505568f577ad service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:29.958411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 705/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:30.838318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d89c6-8756-4b05-ab8d-6d6994cd9ed8 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:30.838953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d89c6-8756-4b05-ab8d-6d6994cd9ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:30.838953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d89c6-8756-4b05-ab8d-6d6994cd9ed8 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:30.840245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:30.845200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6e590-6430-43a2-bbd0-e276e93f05f7 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:30.845662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6e590-6430-43a2-bbd0-e276e93f05f7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:30.845863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6e590-6430-43a2-bbd0-e276e93f05f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:30.865711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:30.922172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-01b6e590-6430-43a2-bbd0-e276e93f05f7 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:30.923374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-01b6e590-6430-43a2-bbd0-e276e93f05f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:31.017049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:31.170481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d89c6-8756-4b05-ab8d-6d6994cd9ed8 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:31.172666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-946d89c6-8756-4b05-ab8d-6d6994cd9ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:31.288402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:31.324714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d89c6-8756-4b05-ab8d-6d6994cd9ed8 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:31.438305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9f484-7069-454a-adcf-1f18bd3fafbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:31.438818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9f484-7069-454a-adcf-1f18bd3fafbf service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:31.439098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9f484-7069-454a-adcf-1f18bd3fafbf service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:31.439368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9f484-7069-454a-adcf-1f18bd3fafbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:31.440968 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-77d9f484-7069-454a-adcf-1f18bd3fafbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:31.515810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:31.529441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9f484-7069-454a-adcf-1f18bd3fafbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:31.531071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9f484-7069-454a-adcf-1f18bd3fafbf service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:31.549370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 719/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:32.426336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e2195-41a0-4297-9c3c-d6409cd8348e None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:32.426922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e2195-41a0-4297-9c3c-d6409cd8348e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:32.426922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e2195-41a0-4297-9c3c-d6409cd8348e None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:32.428259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:32.433603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3d0fe-0edb-4605-bad0-9b15086825da None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:32.434036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3d0fe-0edb-4605-bad0-9b15086825da None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:32.440352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3d0fe-0edb-4605-bad0-9b15086825da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:32.510659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-18f3d0fe-0edb-4605-bad0-9b15086825da None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:32.511876 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-18f3d0fe-0edb-4605-bad0-9b15086825da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:32.609817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:32.764099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e2195-41a0-4297-9c3c-d6409cd8348e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:32.765629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7a5e2195-41a0-4297-9c3c-d6409cd8348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:32.847648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:32.908084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e2195-41a0-4297-9c3c-d6409cd8348e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:33.021177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732f419-8c53-4045-acc1-c67a90fc4d43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:33.022309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f419-8c53-4045-acc1-c67a90fc4d43 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:33.022583 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f419-8c53-4045-acc1-c67a90fc4d43 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:33.022882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f419-8c53-4045-acc1-c67a90fc4d43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:33.024448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7732f419-8c53-4045-acc1-c67a90fc4d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:33.071907 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:33.113584 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732f419-8c53-4045-acc1-c67a90fc4d43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:33.115246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732f419-8c53-4045-acc1-c67a90fc4d43 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:33.134574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 708/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:34.017394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe7c4f-e448-480d-bf88-3afd91c50aa0 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:34.017995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe7c4f-e448-480d-bf88-3afd91c50aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:34.017995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe7c4f-e448-480d-bf88-3afd91c50aa0 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:34.019342 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:34.025027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02a55e-7bfa-4a1e-a83e-36cd19b7d525 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:34.025449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02a55e-7bfa-4a1e-a83e-36cd19b7d525 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:34.025594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02a55e-7bfa-4a1e-a83e-36cd19b7d525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:34.104641 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1f02a55e-7bfa-4a1e-a83e-36cd19b7d525 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:34.105907 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1f02a55e-7bfa-4a1e-a83e-36cd19b7d525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:34.203343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:34.353484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe7c4f-e448-480d-bf88-3afd91c50aa0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:34.355114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-27fe7c4f-e448-480d-bf88-3afd91c50aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:34.404114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:34.501175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe7c4f-e448-480d-bf88-3afd91c50aa0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:34.612612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc36ca-376f-4fdb-a308-dc6255435a52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:34.613211 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc36ca-376f-4fdb-a308-dc6255435a52 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:34.613486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc36ca-376f-4fdb-a308-dc6255435a52 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:34.613791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc36ca-376f-4fdb-a308-dc6255435a52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:34.615892 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c0cc36ca-376f-4fdb-a308-dc6255435a52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:34.622796 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:34.708325 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc36ca-376f-4fdb-a308-dc6255435a52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:34.710146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc36ca-376f-4fdb-a308-dc6255435a52 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:34.728895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 722/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:35.616402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c631dec-41d3-4379-9d42-d63dcc33e106 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:35.616980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c631dec-41d3-4379-9d42-d63dcc33e106 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:35.616980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c631dec-41d3-4379-9d42-d63dcc33e106 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:35.618350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:42:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:35.623434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d440dc4-4e21-40ae-834b-a32f4bba0be2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:35.623836 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d440dc4-4e21-40ae-834b-a32f4bba0be2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:35.623983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d440dc4-4e21-40ae-834b-a32f4bba0be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:35.704670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6d440dc4-4e21-40ae-834b-a32f4bba0be2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:35.705917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6d440dc4-4e21-40ae-834b-a32f4bba0be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:35.781403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:35.811830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:35.964418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c631dec-41d3-4379-9d42-d63dcc33e106 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:35.965894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2c631dec-41d3-4379-9d42-d63dcc33e106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:35.983839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:36.113902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c631dec-41d3-4379-9d42-d63dcc33e106 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:36.206769 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:36.232951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b223d-8480-4387-a635-a4fd447b344d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:36.233460 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b223d-8480-4387-a635-a4fd447b344d service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:36.233820 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b223d-8480-4387-a635-a4fd447b344d service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:36.234029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b223d-8480-4387-a635-a4fd447b344d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:36.235726 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-285b223d-8480-4387-a635-a4fd447b344d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:36.326249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b223d-8480-4387-a635-a4fd447b344d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:36.327851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b223d-8480-4387-a635-a4fd447b344d service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:36.346043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 711/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:37.249379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a20380-3a65-40f7-a3b2-e032081e0e81 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:37.250020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a20380-3a65-40f7-a3b2-e032081e0e81 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:37.250020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a20380-3a65-40f7-a3b2-e032081e0e81 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:37.251501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:37.256359 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28678875-6cd7-4c5f-ab14-657d9354cdd1 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:37.256796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28678875-6cd7-4c5f-ab14-657d9354cdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:37.256894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28678875-6cd7-4c5f-ab14-657d9354cdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:37.332484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-28678875-6cd7-4c5f-ab14-657d9354cdd1 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:37.333715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-28678875-6cd7-4c5f-ab14-657d9354cdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:37.375279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:37.429413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:37.581555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a20380-3a65-40f7-a3b2-e032081e0e81 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:37.583257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-22a20380-3a65-40f7-a3b2-e032081e0e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:37.737917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a20380-3a65-40f7-a3b2-e032081e0e81 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:37.792983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:37.857277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09eeff-e750-4ca5-b154-b01c2f610145 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:37.857817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09eeff-e750-4ca5-b154-b01c2f610145 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:37.858091 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09eeff-e750-4ca5-b154-b01c2f610145 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:37.858357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09eeff-e750-4ca5-b154-b01c2f610145 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:37.859944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fb09eeff-e750-4ca5-b154-b01c2f610145 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:37.952154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09eeff-e750-4ca5-b154-b01c2f610145 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:37.953773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09eeff-e750-4ca5-b154-b01c2f610145 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:37.972387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 725/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:38.833432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf008a57-1022-4b72-97ae-7f9120282c60 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:38.837124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf008a57-1022-4b72-97ae-7f9120282c60 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:38.837124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf008a57-1022-4b72-97ae-7f9120282c60 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:38.837124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:38.840678 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6f70a-b0ab-4cac-8031-5117a8569622 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:38.841088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6f70a-b0ab-4cac-8031-5117a8569622 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:38.841240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6f70a-b0ab-4cac-8031-5117a8569622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:38.925341 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f9e6f70a-b0ab-4cac-8031-5117a8569622 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:38.926589 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f9e6f70a-b0ab-4cac-8031-5117a8569622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:38.929570 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:39.025163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:39.175863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf008a57-1022-4b72-97ae-7f9120282c60 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:39.177293 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bf008a57-1022-4b72-97ae-7f9120282c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:39.323494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf008a57-1022-4b72-97ae-7f9120282c60 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:39.351163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:39.437877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08fd1b-0c70-4dfa-9291-16c8c8fd9c84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:39.438490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08fd1b-0c70-4dfa-9291-16c8c8fd9c84 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:39.438674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08fd1b-0c70-4dfa-9291-16c8c8fd9c84 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:39.438946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08fd1b-0c70-4dfa-9291-16c8c8fd9c84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:39.440576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ac08fd1b-0c70-4dfa-9291-16c8c8fd9c84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:39.530216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08fd1b-0c70-4dfa-9291-16c8c8fd9c84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:39.531802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08fd1b-0c70-4dfa-9291-16c8c8fd9c84 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:39.550198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 714/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:41.487392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4056f-f200-4edd-b24a-90d394d129c0 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:41.487966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4056f-f200-4edd-b24a-90d394d129c0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:41.487966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4056f-f200-4edd-b24a-90d394d129c0 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:41.489356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:41.494777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4923894-60de-4730-8a51-299737eb43fb None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:41.495198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4923894-60de-4730-8a51-299737eb43fb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:41.495311 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4923894-60de-4730-8a51-299737eb43fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:41.536620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:41.561964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e4923894-60de-4730-8a51-299737eb43fb None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:41.563162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e4923894-60de-4730-8a51-299737eb43fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:41.661122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:41.700379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4056f-f200-4edd-b24a-90d394d129c0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:41.701972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-10e4056f-f200-4edd-b24a-90d394d129c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:41.813032 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4056f-f200-4edd-b24a-90d394d129c0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:41.844839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:41.887864 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e28524-dc8f-4965-bcad-e6186358f41a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:41.888470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e28524-dc8f-4965-bcad-e6186358f41a service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:41.888777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e28524-dc8f-4965-bcad-e6186358f41a service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:41.889048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e28524-dc8f-4965-bcad-e6186358f41a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:41.890692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e1e28524-dc8f-4965-bcad-e6186358f41a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:42.009647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e28524-dc8f-4965-bcad-e6186358f41a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:42.011088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e28524-dc8f-4965-bcad-e6186358f41a service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:42.030338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 728/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:44.212498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89c57e-005e-446d-9f8b-d8da5a1cd368 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:44.213027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89c57e-005e-446d-9f8b-d8da5a1cd368 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:44.213027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89c57e-005e-446d-9f8b-d8da5a1cd368 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:44.214342 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:44.219588 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78124e49-e496-48a0-b3c8-bfe74cd6532a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:44.220046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78124e49-e496-48a0-b3c8-bfe74cd6532a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:44.220151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78124e49-e496-48a0-b3c8-bfe74cd6532a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:44.287144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-78124e49-e496-48a0-b3c8-bfe74cd6532a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:44.288348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-78124e49-e496-48a0-b3c8-bfe74cd6532a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:44.300807 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:44.386747 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:44.424885 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89c57e-005e-446d-9f8b-d8da5a1cd368 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:44.426440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dc89c57e-005e-446d-9f8b-d8da5a1cd368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:44.532361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89c57e-005e-446d-9f8b-d8da5a1cd368 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:44.544414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:44.608025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5194d-a158-4b47-a14a-df9497c5e3d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:44.608631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5194d-a158-4b47-a14a-df9497c5e3d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:44.608910 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5194d-a158-4b47-a14a-df9497c5e3d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:44.609183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5194d-a158-4b47-a14a-df9497c5e3d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:44.610828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a1c5194d-a158-4b47-a14a-df9497c5e3d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:44.720137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5194d-a158-4b47-a14a-df9497c5e3d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:44.721587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5194d-a158-4b47-a14a-df9497c5e3d0 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:44.741298 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 717/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:47.162559 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304d7d-341f-4c12-928f-8e1374bea870 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:47.163156 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304d7d-341f-4c12-928f-8e1374bea870 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:47.163156 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304d7d-341f-4c12-928f-8e1374bea870 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:47.164473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:47.169965 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495f920-a253-414e-a926-de80138c15e5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:47.170401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495f920-a253-414e-a926-de80138c15e5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:47.170497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495f920-a253-414e-a926-de80138c15e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:47.220553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:47.239504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0495f920-a253-414e-a926-de80138c15e5 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:47.241306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0495f920-a253-414e-a926-de80138c15e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:47.339712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:47.379452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d304d7d-341f-4c12-928f-8e1374bea870 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:47.381005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4d304d7d-341f-4c12-928f-8e1374bea870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:47.487848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d304d7d-341f-4c12-928f-8e1374bea870 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:47.529235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:47.563710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb49fc-f62c-4843-a38e-ec33b332299a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:47.564314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb49fc-f62c-4843-a38e-ec33b332299a service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:47.564587 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb49fc-f62c-4843-a38e-ec33b332299a service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:47.564859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb49fc-f62c-4843-a38e-ec33b332299a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:47.566473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-aeeb49fc-f62c-4843-a38e-ec33b332299a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:47.676167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb49fc-f62c-4843-a38e-ec33b332299a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:47.678297 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb49fc-f62c-4843-a38e-ec33b332299a service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:47.696927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 731/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:49.871628 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecad065-89e6-4ca9-af27-e97eaa1d4380 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:49.872209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecad065-89e6-4ca9-af27-e97eaa1d4380 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:49.872209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecad065-89e6-4ca9-af27-e97eaa1d4380 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:49.873635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:42:49.878910 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d85e5-8d0d-43b8-a1c8-cb97617ab9ba None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:49.879329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d85e5-8d0d-43b8-a1c8-cb97617ab9ba None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:49.879440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d85e5-8d0d-43b8-a1c8-cb97617ab9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:49.950339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a84d85e5-8d0d-43b8-a1c8-cb97617ab9ba None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:49.951564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a84d85e5-8d0d-43b8-a1c8-cb97617ab9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:49.972313 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:50.049112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:50.088378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecad065-89e6-4ca9-af27-e97eaa1d4380 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:50.090028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6ecad065-89e6-4ca9-af27-e97eaa1d4380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:50.194723 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecad065-89e6-4ca9-af27-e97eaa1d4380 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:50.214686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:50.272991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df47b0-4e38-45b5-94e0-d9f750ad0268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:50.273870 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df47b0-4e38-45b5-94e0-d9f750ad0268 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:50.274604 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df47b0-4e38-45b5-94e0-d9f750ad0268 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:50.274887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df47b0-4e38-45b5-94e0-d9f750ad0268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:50.276518 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-15df47b0-4e38-45b5-94e0-d9f750ad0268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:50.387918 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df47b0-4e38-45b5-94e0-d9f750ad0268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:50.389408 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df47b0-4e38-45b5-94e0-d9f750ad0268 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:50.408205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 720/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:52.476693 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd189fc-abed-433d-8252-9e61d9b9cfbf None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:52.476693 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd189fc-abed-433d-8252-9e61d9b9cfbf None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:52.477303 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd189fc-abed-433d-8252-9e61d9b9cfbf None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:52.478161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:42:52.483575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3653d-652a-4443-b68b-10094de7adfd None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:52.484138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3653d-652a-4443-b68b-10094de7adfd None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:52.484417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3653d-652a-4443-b68b-10094de7adfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:52.548149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:52.551302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4fe3653d-652a-4443-b68b-10094de7adfd None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:52.552555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4fe3653d-652a-4443-b68b-10094de7adfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:52.650129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:52.688118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd189fc-abed-433d-8252-9e61d9b9cfbf None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:52.689723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9bd189fc-abed-433d-8252-9e61d9b9cfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:52.795221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd189fc-abed-433d-8252-9e61d9b9cfbf None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:52.845469 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:52.870024 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c799b3-5ae1-4ca3-911b-ee1588954871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:52.870623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c799b3-5ae1-4ca3-911b-ee1588954871 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:52.870894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c799b3-5ae1-4ca3-911b-ee1588954871 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:52.871160 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c799b3-5ae1-4ca3-911b-ee1588954871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:52.872741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b8c799b3-5ae1-4ca3-911b-ee1588954871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:52.983052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c799b3-5ae1-4ca3-911b-ee1588954871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:52.984476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c799b3-5ae1-4ca3-911b-ee1588954871 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:53.001980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 734/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:55.085245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f456490-9659-4943-a7b5-29435834b12d None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:55.085816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f456490-9659-4943-a7b5-29435834b12d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:55.085816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f456490-9659-4943-a7b5-29435834b12d None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:55.087182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:42:55.093062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0585fc8-de2d-4b4c-a2aa-1bd08dc8647d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:55.093592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0585fc8-de2d-4b4c-a2aa-1bd08dc8647d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:55.093705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0585fc8-de2d-4b4c-a2aa-1bd08dc8647d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:55.163071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c0585fc8-de2d-4b4c-a2aa-1bd08dc8647d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:55.164294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c0585fc8-de2d-4b4c-a2aa-1bd08dc8647d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:55.195176 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:55.264498 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:55.303927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f456490-9659-4943-a7b5-29435834b12d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:55.305448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2f456490-9659-4943-a7b5-29435834b12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:55.408066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f456490-9659-4943-a7b5-29435834b12d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:55.437303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:55.484841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b04929-809d-4c4b-80ce-b0d0b4319deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:55.485443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b04929-809d-4c4b-80ce-b0d0b4319deb service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:55.485754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b04929-809d-4c4b-80ce-b0d0b4319deb service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:55.486030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b04929-809d-4c4b-80ce-b0d0b4319deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:55.487649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b8b04929-809d-4c4b-80ce-b0d0b4319deb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:55.599725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b04929-809d-4c4b-80ce-b0d0b4319deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:55.601161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b04929-809d-4c4b-80ce-b0d0b4319deb service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:55.620162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 723/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:42:57.738533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0210b1-ccf1-4777-ad71-03d9da0f9123 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:57.739076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0210b1-ccf1-4777-ad71-03d9da0f9123 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:57.739076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0210b1-ccf1-4777-ad71-03d9da0f9123 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:57.741315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:42:57.746724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418edf2-3649-42c9-8580-9107d0eaa8db None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:57.747145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418edf2-3649-42c9-8580-9107d0eaa8db None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:57.747277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418edf2-3649-42c9-8580-9107d0eaa8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:57.814064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4418edf2-3649-42c9-8580-9107d0eaa8db None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:42:57.815281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4418edf2-3649-42c9-8580-9107d0eaa8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:57.824153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:57.917499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:42:57.955449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0210b1-ccf1-4777-ad71-03d9da0f9123 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:42:57.957054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1e0210b1-ccf1-4777-ad71-03d9da0f9123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:58.069945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0210b1-ccf1-4777-ad71-03d9da0f9123 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:42:58.127414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:42:58.147463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48134e-86bb-4797-9e61-74d421ca8d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:42:58.148071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48134e-86bb-4797-9e61-74d421ca8d48 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:42:58.148378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48134e-86bb-4797-9e61-74d421ca8d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:42:58.148655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48134e-86bb-4797-9e61-74d421ca8d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:42:58.150322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cf48134e-86bb-4797-9e61-74d421ca8d48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:42:58.273134 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48134e-86bb-4797-9e61-74d421ca8d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:42:58.274666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48134e-86bb-4797-9e61-74d421ca8d48 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:42:58.293234 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 737/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:42:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:43:00.362680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bdc67-3ab9-4053-a5ec-e2c9050cdf8b None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:00.363258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bdc67-3ab9-4053-a5ec-e2c9050cdf8b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:00.363258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bdc67-3ab9-4053-a5ec-e2c9050cdf8b None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:00.364674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:43:00.369959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dcf2f-ec7c-4e19-8374-a01e4af2fe2f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:00.370330 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dcf2f-ec7c-4e19-8374-a01e4af2fe2f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:00.370509 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dcf2f-ec7c-4e19-8374-a01e4af2fe2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:00.435806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-559dcf2f-ec7c-4e19-8374-a01e4af2fe2f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:43:00.436998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-559dcf2f-ec7c-4e19-8374-a01e4af2fe2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:00.475007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:00.532947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:43:00.570675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bdc67-3ab9-4053-a5ec-e2c9050cdf8b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:00.572216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-da6bdc67-3ab9-4053-a5ec-e2c9050cdf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:00.674789 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bdc67-3ab9-4053-a5ec-e2c9050cdf8b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:00.712410 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:00.747857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50038e0c-00ff-401e-8cca-1819aa85db63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:00.748458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50038e0c-00ff-401e-8cca-1819aa85db63 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:00.748736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50038e0c-00ff-401e-8cca-1819aa85db63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:00.749005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50038e0c-00ff-401e-8cca-1819aa85db63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:00.750617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-50038e0c-00ff-401e-8cca-1819aa85db63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:00.859055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50038e0c-00ff-401e-8cca-1819aa85db63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:00.860474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50038e0c-00ff-401e-8cca-1819aa85db63 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:00.878894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 726/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:43:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:43:02.918815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b580c-52e5-4d6b-9e4c-870a8149ec85 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:02.919437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b580c-52e5-4d6b-9e4c-870a8149ec85 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:02.919437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b580c-52e5-4d6b-9e4c-870a8149ec85 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:02.920773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:43:02.926105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879e952-01b2-442e-a399-39ebbcc7e609 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:02.926585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879e952-01b2-442e-a399-39ebbcc7e609 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:02.927253 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879e952-01b2-442e-a399-39ebbcc7e609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:02.997843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5879e952-01b2-442e-a399-39ebbcc7e609 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:43:02.999056 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5879e952-01b2-442e-a399-39ebbcc7e609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:03.016677 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:03.097872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:43:03.138186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b580c-52e5-4d6b-9e4c-870a8149ec85 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:03.139731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-213b580c-52e5-4d6b-9e4c-870a8149ec85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:03.247754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b580c-52e5-4d6b-9e4c-870a8149ec85 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:03.318171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:03.325240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661be161-e82e-4cc3-9c5e-c4734fd14b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:03.328199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661be161-e82e-4cc3-9c5e-c4734fd14b0f service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:03.328199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661be161-e82e-4cc3-9c5e-c4734fd14b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:03.328199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661be161-e82e-4cc3-9c5e-c4734fd14b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:03.328199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-661be161-e82e-4cc3-9c5e-c4734fd14b0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:03.438051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661be161-e82e-4cc3-9c5e-c4734fd14b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:03.439503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661be161-e82e-4cc3-9c5e-c4734fd14b0f service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:03.460955 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 740/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:43:05.479224 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d9a78-4efc-403b-80db-cd3b873598ed None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:05.479812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d9a78-4efc-403b-80db-cd3b873598ed None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:05.479812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d9a78-4efc-403b-80db-cd3b873598ed None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:05.481098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:43:05.486291 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57a37e-90c9-4983-b800-54803e3b831f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:05.486701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57a37e-90c9-4983-b800-54803e3b831f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:05.486831 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57a37e-90c9-4983-b800-54803e3b831f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:05.554287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5b57a37e-90c9-4983-b800-54803e3b831f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:43:05.555401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5b57a37e-90c9-4983-b800-54803e3b831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:05.603547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:05.652581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:43:05.689834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d9a78-4efc-403b-80db-cd3b873598ed None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:05.691401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7f1d9a78-4efc-403b-80db-cd3b873598ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:05.798625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d9a78-4efc-403b-80db-cd3b873598ed None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:05.850267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:05.876850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621eddc-7fcd-4a2c-a19e-8435fa79f0c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:05.877453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621eddc-7fcd-4a2c-a19e-8435fa79f0c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:05.877832 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621eddc-7fcd-4a2c-a19e-8435fa79f0c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:05.878104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621eddc-7fcd-4a2c-a19e-8435fa79f0c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:05.879714 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7621eddc-7fcd-4a2c-a19e-8435fa79f0c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:06.005421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621eddc-7fcd-4a2c-a19e-8435fa79f0c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:06.006996 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621eddc-7fcd-4a2c-a19e-8435fa79f0c1 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:06.028026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 729/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:43:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:43:15.650945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=22 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:16.025772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=66 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:25.771157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454e553-e967-4efe-9e14-f36ea63e9cc8 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:25.771747 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454e553-e967-4efe-9e14-f36ea63e9cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:25.771747 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454e553-e967-4efe-9e14-f36ea63e9cc8 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:25.773101 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 742/1471] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:43:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:43:25.778354 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede457-1a07-43e1-a939-0290bee452ea None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:25.778829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede457-1a07-43e1-a939-0290bee452ea None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:25.778942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede457-1a07-43e1-a939-0290bee452ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:25.858235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5eede457-1a07-43e1-a939-0290bee452ea None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:43:25.859450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5eede457-1a07-43e1-a939-0290bee452ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:25.956037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 730/1472] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:43:26.041327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4af68-62b8-4b50-8c18-ad507376f1f6 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:26.041656 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4af68-62b8-4b50-8c18-ad507376f1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:26.041932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4af68-62b8-4b50-8c18-ad507376f1f6 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:26.043266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 743/1473] 10.222.0.57 () {54 vars in 967 bytes} [Tue Aug 9 18:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:43:26.048028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771e20a-f6d7-434d-9b4a-6090e281be63 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:26.048314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771e20a-f6d7-434d-9b4a-6090e281be63 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:26.048591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771e20a-f6d7-434d-9b4a-6090e281be63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:26.124710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6771e20a-f6d7-434d-9b4a-6090e281be63 None None] MFA Rules not processed for user `5a58f9101f2d4785b96f0cc55be4330b`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:43:26.125973 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6771e20a-f6d7-434d-9b4a-6090e281be63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:26.148159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:26.206337 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 731/1474] 10.222.0.57 () {58 vars in 1064 bytes} [Tue Aug 9 18:43:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:43:26.214488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4af68-62b8-4b50-8c18-ad507376f1f6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:26.216072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-06f4af68-62b8-4b50-8c18-ad507376f1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:26.346858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4af68-62b8-4b50-8c18-ad507376f1f6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:26.422763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bc3ea-fc89-46cc-8368-79cbe9b03127 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a58f9101f2d4785b96f0cc55be4330b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:26.427659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bc3ea-fc89-46cc-8368-79cbe9b03127 service glance] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:26.427659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bc3ea-fc89-46cc-8368-79cbe9b03127 service glance] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:26.427659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bc3ea-fc89-46cc-8368-79cbe9b03127 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:26.427659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a70bc3ea-fc89-46cc-8368-79cbe9b03127 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:26.479949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:26.535230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70bc3ea-fc89-46cc-8368-79cbe9b03127 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:26.536673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70bc3ea-fc89-46cc-8368-79cbe9b03127 service glance] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:26.555642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 744/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:43:28.716093 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97793191-ca9a-46e4-b986-571e151d46d4 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:28.716684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97793191-ca9a-46e4-b986-571e151d46d4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:28.716684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97793191-ca9a-46e4-b986-571e151d46d4 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:28.718013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 732/1476] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:43:28.723336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd300612-2622-4a9d-beba-63398b0f8fdd None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:28.723785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd300612-2622-4a9d-beba-63398b0f8fdd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:28.723901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd300612-2622-4a9d-beba-63398b0f8fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:28.794458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-fd300612-2622-4a9d-beba-63398b0f8fdd None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:43:28.795688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fd300612-2622-4a9d-beba-63398b0f8fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:28.899690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 745/1477] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:43:28.920696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97793191-ca9a-46e4-b986-571e151d46d4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:28.922255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-97793191-ca9a-46e4-b986-571e151d46d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:29.049245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97793191-ca9a-46e4-b986-571e151d46d4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:29.099848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:29.143908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6154071-bc7f-4ebe-ad02-e9831f1f9fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:29.144506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6154071-bc7f-4ebe-ad02-e9831f1f9fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:29.144780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6154071-bc7f-4ebe-ad02-e9831f1f9fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:29.145050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6154071-bc7f-4ebe-ad02-e9831f1f9fb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:29.145713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6154071-bc7f-4ebe-ad02-e9831f1f9fb8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:29.147446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-d6154071-bc7f-4ebe-ad02-e9831f1f9fb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:43:29.147446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Aug 09 18:43:29.147446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: Aug 09 18:43:29.148573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6154071-bc7f-4ebe-ad02-e9831f1f9fb8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:29.168238 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 733/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Tue Aug 9 18:43:28 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:43:31.230871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff6808-f136-4899-8e47-35f6da5f9458 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:31.231406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff6808-f136-4899-8e47-35f6da5f9458 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:31.231523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff6808-f136-4899-8e47-35f6da5f9458 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:31.232798 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 746/1479] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:43:31.238003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239667ed-31f3-4705-a6ab-50cdb58fda85 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:31.240057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239667ed-31f3-4705-a6ab-50cdb58fda85 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:31.240057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239667ed-31f3-4705-a6ab-50cdb58fda85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:31.310146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-239667ed-31f3-4705-a6ab-50cdb58fda85 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:43:31.311419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-239667ed-31f3-4705-a6ab-50cdb58fda85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:31.407852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 734/1480] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:43:31.452669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff6808-f136-4899-8e47-35f6da5f9458 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:31.454234 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-37ff6808-f136-4899-8e47-35f6da5f9458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:31.464073 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:31.601436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff6808-f136-4899-8e47-35f6da5f9458 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:31.681736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:31.714816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492fd91-a314-412c-b600-fa6a048027b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:31.715342 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492fd91-a314-412c-b600-fa6a048027b3 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:31.715621 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492fd91-a314-412c-b600-fa6a048027b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:31.715892 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492fd91-a314-412c-b600-fa6a048027b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:31.718279 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1492fd91-a314-412c-b600-fa6a048027b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:31.811725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492fd91-a314-412c-b600-fa6a048027b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:31.813327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492fd91-a314-412c-b600-fa6a048027b3 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:31.832467 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 747/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:43:34.051509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d9a42-c598-458f-a067-6096a0f6d188 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:34.052073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d9a42-c598-458f-a067-6096a0f6d188 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:34.052073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d9a42-c598-458f-a067-6096a0f6d188 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:34.053720 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 735/1482] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:43:34.058952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d7d69-73da-4b3e-9bcc-f26f33bb38ca None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:34.063123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d7d69-73da-4b3e-9bcc-f26f33bb38ca None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:34.063123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d7d69-73da-4b3e-9bcc-f26f33bb38ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:34.130277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-9f8d7d69-73da-4b3e-9bcc-f26f33bb38ca None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:43:34.131414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9f8d7d69-73da-4b3e-9bcc-f26f33bb38ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:34.179122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:34.230631 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 748/1483] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:43:34.267664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d9a42-c598-458f-a067-6096a0f6d188 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:34.269230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8a2d9a42-c598-458f-a067-6096a0f6d188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:34.298733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:34.380791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d9a42-c598-458f-a067-6096a0f6d188 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:34.459386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b5c35-b3da-4c1d-943f-1e722f81b797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:34.459990 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b5c35-b3da-4c1d-943f-1e722f81b797 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:34.460268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b5c35-b3da-4c1d-943f-1e722f81b797 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:34.460540 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b5c35-b3da-4c1d-943f-1e722f81b797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:34.462243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f81b5c35-b3da-4c1d-943f-1e722f81b797 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:34.537430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:34.571219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b5c35-b3da-4c1d-943f-1e722f81b797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:34.572645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b5c35-b3da-4c1d-943f-1e722f81b797 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:34.592148 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 736/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:43:36.656765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae8028-ad0e-4ce2-ae4d-31353706efc7 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:36.657330 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae8028-ad0e-4ce2-ae4d-31353706efc7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:36.657428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae8028-ad0e-4ce2-ae4d-31353706efc7 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:36.658925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 749/1485] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:43:36.665658 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613341a7-71dd-4cec-bb21-f4075e89d734 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:36.673722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613341a7-71dd-4cec-bb21-f4075e89d734 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:36.673722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613341a7-71dd-4cec-bb21-f4075e89d734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:36.733417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-613341a7-71dd-4cec-bb21-f4075e89d734 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:43:36.734673 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-613341a7-71dd-4cec-bb21-f4075e89d734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:36.832592 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 737/1486] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:43:36.871778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae8028-ad0e-4ce2-ae4d-31353706efc7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:36.873361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c1ae8028-ad0e-4ce2-ae4d-31353706efc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:36.979429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae8028-ad0e-4ce2-ae4d-31353706efc7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:37.051283 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:37.053046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e27193-e36b-499a-8acc-0ca1c6413b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:37.053982 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27193-e36b-499a-8acc-0ca1c6413b52 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:37.054265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27193-e36b-499a-8acc-0ca1c6413b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:37.054540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27193-e36b-499a-8acc-0ca1c6413b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:37.056168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-32e27193-e36b-499a-8acc-0ca1c6413b52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:37.166538 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e27193-e36b-499a-8acc-0ca1c6413b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:37.167898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e27193-e36b-499a-8acc-0ca1c6413b52 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:37.186630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 750/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:43:39.344497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b814cb6-7b2c-4403-afba-091b0a1477f2 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:39.345095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b814cb6-7b2c-4403-afba-091b0a1477f2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:39.345095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b814cb6-7b2c-4403-afba-091b0a1477f2 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:39.346440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 738/1488] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:43:39.351663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a384de-70af-4cf1-982a-fd418b9030e0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:39.352146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a384de-70af-4cf1-982a-fd418b9030e0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:39.352240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a384de-70af-4cf1-982a-fd418b9030e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:39.426077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-01a384de-70af-4cf1-982a-fd418b9030e0 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:43:39.427286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-01a384de-70af-4cf1-982a-fd418b9030e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:39.480405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:39.525143 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 751/1489] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:43:39.566595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b814cb6-7b2c-4403-afba-091b0a1477f2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:39.568157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9b814cb6-7b2c-4403-afba-091b0a1477f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:39.603762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:39.682133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b814cb6-7b2c-4403-afba-091b0a1477f2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:39.766983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03ee3b-2117-4f1c-a171-c7a6d51737db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:39.767577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03ee3b-2117-4f1c-a171-c7a6d51737db service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:39.767856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03ee3b-2117-4f1c-a171-c7a6d51737db service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:39.768124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03ee3b-2117-4f1c-a171-c7a6d51737db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:39.770392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ce03ee3b-2117-4f1c-a171-c7a6d51737db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:39.851764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:39.880157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03ee3b-2117-4f1c-a171-c7a6d51737db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:39.881662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03ee3b-2117-4f1c-a171-c7a6d51737db service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:39.900634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 739/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:43:42.632911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd8b73-c34b-47da-8785-49d3902090c8 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:42.633546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd8b73-c34b-47da-8785-49d3902090c8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:42.633546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd8b73-c34b-47da-8785-49d3902090c8 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:42.634907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 752/1491] 10.222.0.57 () {54 vars in 927 bytes} [Tue Aug 9 18:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:43:42.640093 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f8c5c-f82a-4131-ae45-e75c8f96a380 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:42.640567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f8c5c-f82a-4131-ae45-e75c8f96a380 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:42.640683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f8c5c-f82a-4131-ae45-e75c8f96a380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:42.707359 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e04f8c5c-f82a-4131-ae45-e75c8f96a380 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:43:42.708618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e04f8c5c-f82a-4131-ae45-e75c8f96a380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:42.806134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 740/1492] 10.222.0.57 () {58 vars in 1024 bytes} [Tue Aug 9 18:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:43:42.845830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd8b73-c34b-47da-8785-49d3902090c8 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:43:42.848003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-92cd8b73-c34b-47da-8785-49d3902090c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:42.950722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd8b73-c34b-47da-8785-49d3902090c8 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:43:43.024266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5d0bb-1791-4b58-ae81-bcb0a86b1390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:43:43.024881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5d0bb-1791-4b58-ae81-bcb0a86b1390 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:43:43.025162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5d0bb-1791-4b58-ae81-bcb0a86b1390 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:43:43.025433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5d0bb-1791-4b58-ae81-bcb0a86b1390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:43:43.027077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d6b5d0bb-1791-4b58-ae81-bcb0a86b1390 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:43:43.036928 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:43.141314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5d0bb-1791-4b58-ae81-bcb0a86b1390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:43:43.142820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5d0bb-1791-4b58-ae81-bcb0a86b1390 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:43:43.162197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 753/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:43:52.804263 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=82 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:43:53.159434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=50 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:45:11.932289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb505d34-2fea-4a48-bf77-ba800295d291 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:11.932904 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb505d34-2fea-4a48-bf77-ba800295d291 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:11.932904 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb505d34-2fea-4a48-bf77-ba800295d291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:12.009260 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-bb505d34-2fea-4a48-bf77-ba800295d291 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:45:12.010513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bb505d34-2fea-4a48-bf77-ba800295d291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:12.107237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 741/1494] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:45:12.118470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d06997-6a92-49c4-b705-ceb2a180f2cc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:12.118998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d06997-6a92-49c4-b705-ceb2a180f2cc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:12.118998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d06997-6a92-49c4-b705-ceb2a180f2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:12.195762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d8d06997-6a92-49c4-b705-ceb2a180f2cc None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:45:12.196885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d8d06997-6a92-49c4-b705-ceb2a180f2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:12.296172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 754/1495] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:45:12.301474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb505d34-2fea-4a48-bf77-ba800295d291 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:45:12.303068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bb505d34-2fea-4a48-bf77-ba800295d291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:12.314513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb505d34-2fea-4a48-bf77-ba800295d291 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:45:12.376732 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:45:12.413131 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec615c-9140-453c-a3c7-49c66d30acf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:45:12.414896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec615c-9140-453c-a3c7-49c66d30acf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:12.414896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec615c-9140-453c-a3c7-49c66d30acf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:12.414896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec615c-9140-453c-a3c7-49c66d30acf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:12.415337 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec615c-9140-453c-a3c7-49c66d30acf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:45:12.415474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec615c-9140-453c-a3c7-49c66d30acf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:45:12.416555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-79ec615c-9140-453c-a3c7-49c66d30acf9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:45:12.416555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Aug 09 18:45:12.416555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: Aug 09 18:45:12.417557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec615c-9140-453c-a3c7-49c66d30acf9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:45:12.431676 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 742/1496] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:45:12.440450 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d06997-6a92-49c4-b705-ceb2a180f2cc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:45:12.449585 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d06997-6a92-49c4-b705-ceb2a180f2cc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:45:12.512911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:45:12.545059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3834e6-34cf-45be-aaae-2b997a612c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:45:12.545680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3834e6-34cf-45be-aaae-2b997a612c1b admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:12.545963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3834e6-34cf-45be-aaae-2b997a612c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:12.546724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3834e6-34cf-45be-aaae-2b997a612c1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:12.547353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3834e6-34cf-45be-aaae-2b997a612c1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:45:12.549827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-5e3834e6-34cf-45be-aaae-2b997a612c1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:45:12.549827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Aug 09 18:45:12.549827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: Aug 09 18:45:12.550994 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3834e6-34cf-45be-aaae-2b997a612c1b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:45:12.687704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 755/1497] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:45:12 2022] POST /identity/v3/projects => generated 379 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:45:12.692266 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec615c-9140-453c-a3c7-49c66d30acf9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:45:12.693836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-79ec615c-9140-453c-a3c7-49c66d30acf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:12.728496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec615c-9140-453c-a3c7-49c66d30acf9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:45:12.822308 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807b819-9652-4cf9-82f5-5ec0dd345cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:45:12.822910 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807b819-9652-4cf9-82f5-5ec0dd345cfa admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:12.823190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807b819-9652-4cf9-82f5-5ec0dd345cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:12.823463 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807b819-9652-4cf9-82f5-5ec0dd345cfa admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:12.824095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807b819-9652-4cf9-82f5-5ec0dd345cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:45:12.825157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-7807b819-9652-4cf9-82f5-5ec0dd345cfa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:45:12.825157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Aug 09 18:45:12.825157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: Aug 09 18:45:12.826368 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807b819-9652-4cf9-82f5-5ec0dd345cfa admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:45:12.961092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 743/1498] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:45:12 2022] POST /identity/v3/users => generated 290 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:45:12.965456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3834e6-34cf-45be-aaae-2b997a612c1b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:45:12.967038 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5e3834e6-34cf-45be-aaae-2b997a612c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:13.029370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3834e6-34cf-45be-aaae-2b997a612c1b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:45:13.123180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:45:13.140963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70825d-6b4f-4c1d-9c39-454b7628943d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:45:13.141603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70825d-6b4f-4c1d-9c39-454b7628943d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:13.141888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70825d-6b4f-4c1d-9c39-454b7628943d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:13.142159 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70825d-6b4f-4c1d-9c39-454b7628943d admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:13.142816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70825d-6b4f-4c1d-9c39-454b7628943d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:45:13.144015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-fb70825d-6b4f-4c1d-9c39-454b7628943d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:45:13.144015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Aug 09 18:45:13.144015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: Aug 09 18:45:13.144915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70825d-6b4f-4c1d-9c39-454b7628943d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:45:13.160219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 756/1499] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:45:13.164551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807b819-9652-4cf9-82f5-5ec0dd345cfa admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:45:13.166145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7807b819-9652-4cf9-82f5-5ec0dd345cfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:13.178670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807b819-9652-4cf9-82f5-5ec0dd345cfa admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:45:13.206361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:45:13.286673 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11c922-b31f-4b4b-bbcf-9ca9535421d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:45:13.287231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11c922-b31f-4b4b-bbcf-9ca9535421d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:13.287506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11c922-b31f-4b4b-bbcf-9ca9535421d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:13.287780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11c922-b31f-4b4b-bbcf-9ca9535421d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:13.288422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11c922-b31f-4b4b-bbcf-9ca9535421d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:45:13.289432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-ed11c922-b31f-4b4b-bbcf-9ca9535421d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:45:13.289432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Aug 09 18:45:13.289432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: Aug 09 18:45:13.290434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11c922-b31f-4b4b-bbcf-9ca9535421d2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:45:13.305770 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 744/1500] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:45:13.310051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70825d-6b4f-4c1d-9c39-454b7628943d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:45:13.311574 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fb70825d-6b4f-4c1d-9c39-454b7628943d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:13.348629 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70825d-6b4f-4c1d-9c39-454b7628943d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:45:13.459538 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b4de7-cc8e-425d-bd80-b49368d025d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:45:13.460217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b4de7-cc8e-425d-bd80-b49368d025d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:13.460499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b4de7-cc8e-425d-bd80-b49368d025d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:13.460772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b4de7-cc8e-425d-bd80-b49368d025d7 admin admin] PATH_INFO: `/v3/projects/69b1c7093d4e453087bc8fde7e561430/users/d286aae4986043b0b4435bdedcd09220/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:13.486768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b4de7-cc8e-425d-bd80-b49368d025d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b1c7093d4e453087bc8fde7e561430, user_id=d286aae4986043b0b4435bdedcd09220, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:45:13.487812 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-ad4b4de7-cc8e-425d-bd80-b49368d025d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:45:13.487812 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Aug 09 18:45:13.487812 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: Aug 09 18:45:13.489814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b4de7-cc8e-425d-bd80-b49368d025d7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:45:13.586528 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 757/1501] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:45:13 2022] PUT /identity/v3/projects/69b1c7093d4e453087bc8fde7e561430/users/d286aae4986043b0b4435bdedcd09220/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:45:13.599678 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bfd95-bcee-4c76-b941-ad034338573c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:13.599959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bfd95-bcee-4c76-b941-ad034338573c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:13.600255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bfd95-bcee-4c76-b941-ad034338573c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:13.658209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-018bfd95-bcee-4c76-b941-ad034338573c None None] MFA Rules not processed for user `d286aae4986043b0b4435bdedcd09220`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:45:13.659422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-018bfd95-bcee-4c76-b941-ad034338573c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:13.738711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 745/1502] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:45:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:45:13.752671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c98b9-9cbc-408e-a69a-040ddf09d5ec None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:13.753094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c98b9-9cbc-408e-a69a-040ddf09d5ec None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:13.753375 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c98b9-9cbc-408e-a69a-040ddf09d5ec None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:13.754762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 758/1503] 10.222.0.57 () {54 vars in 964 bytes} [Tue Aug 9 18:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:45:13.760129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab269b5-c9bc-48b3-b5de-f6696126b02c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:13.760440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab269b5-c9bc-48b3-b5de-f6696126b02c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:13.760719 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab269b5-c9bc-48b3-b5de-f6696126b02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:13.794990 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:45:13.847526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4ab269b5-c9bc-48b3-b5de-f6696126b02c None None] MFA Rules not processed for user `77300e25b9004a598f8f7f12f3cf4866`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:45:13.848663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4ab269b5-c9bc-48b3-b5de-f6696126b02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:13.933671 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 746/1504] 10.222.0.57 () {58 vars in 1061 bytes} [Tue Aug 9 18:45:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:45:13.942223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c98b9-9cbc-408e-a69a-040ddf09d5ec None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:45:13.943760 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ba0c98b9-9cbc-408e-a69a-040ddf09d5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:13.996384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:45:14.050930 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c98b9-9cbc-408e-a69a-040ddf09d5ec None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:45:14.135471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc8b68-e91d-4658-aafa-619156e05b5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:45:14.136143 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8b68-e91d-4658-aafa-619156e05b5d service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:14.136257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8b68-e91d-4658-aafa-619156e05b5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:14.136515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8b68-e91d-4658-aafa-619156e05b5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:14.138154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-78cc8b68-e91d-4658-aafa-619156e05b5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:14.237070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc8b68-e91d-4658-aafa-619156e05b5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:45:14.238527 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc8b68-e91d-4658-aafa-619156e05b5d service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:45:14.241985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:45:14.257225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 759/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:45:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:45:14.303231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847daaf4-ef21-47db-b778-85a706e8c8ca None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:14.303647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847daaf4-ef21-47db-b778-85a706e8c8ca None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:14.303771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847daaf4-ef21-47db-b778-85a706e8c8ca None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:14.305050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 747/1506] 10.222.0.57 () {52 vars in 830 bytes} [Tue Aug 9 18:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:45:14.308617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc8b68-e91d-4658-aafa-619156e05b5d service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:45:14.309881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-78cc8b68-e91d-4658-aafa-619156e05b5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:14.409054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc8b68-e91d-4658-aafa-619156e05b5d service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:45:14.503276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:45:14.503928 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:14.504209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:14.504480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] PATH_INFO: `/v3/users/d286aae4986043b0b4435bdedcd09220` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:14.516770 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d286aae4986043b0b4435bdedcd09220)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:45:14.517816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:45:14.517816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Aug 09 18:45:14.517816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: Aug 09 18:45:14.519037 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:45:14.528952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:45:14.750849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd286aae4986043b0b4435bdedcd09220'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:45:14.824614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd286aae4986043b0b4435bdedcd09220'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:45:14.825028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd286aae4986043b0b4435bdedcd09220'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:45:14.839746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd286aae4986043b0b4435bdedcd09220'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:45:14.863921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd286aae4986043b0b4435bdedcd09220'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:45:14.872046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 760/1507] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:45:14 2022] DELETE /identity/v3/users/d286aae4986043b0b4435bdedcd09220 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:45:14.876425 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910be017-7bde-4893-ab12-1385608f8420 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:14.876710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910be017-7bde-4893-ab12-1385608f8420 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:14.877008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910be017-7bde-4893-ab12-1385608f8420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:14.936500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-910be017-7bde-4893-ab12-1385608f8420 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:45:14.937858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-910be017-7bde-4893-ab12-1385608f8420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:15.013599 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:45:15.026413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 748/1508] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:45:15.036565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:45:15.037818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:15.137264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78b54b-3e54-4b49-8651-42c6f6975a3f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:45:15.230092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:45:15.249696 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ca7f5-bb6a-4a16-a437-b0357ff0bb7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:45:15.250434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ca7f5-bb6a-4a16-a437-b0357ff0bb7a service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:15.250718 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ca7f5-bb6a-4a16-a437-b0357ff0bb7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:15.250993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ca7f5-bb6a-4a16-a437-b0357ff0bb7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:15.252621 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a16ca7f5-bb6a-4a16-a437-b0357ff0bb7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:15.364449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ca7f5-bb6a-4a16-a437-b0357ff0bb7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:45:15.365936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ca7f5-bb6a-4a16-a437-b0357ff0bb7a service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:45:15.384993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 761/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:45:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:45:15.921842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910be017-7bde-4893-ab12-1385608f8420 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:45:15.923152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-910be017-7bde-4893-ab12-1385608f8420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:45:15.934868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910be017-7bde-4893-ab12-1385608f8420 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:45:16.035126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:45:16.035760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:45:16.036042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:45:16.036318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] PATH_INFO: `/v3/projects/69b1c7093d4e453087bc8fde7e561430` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:45:16.047050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b1c7093d4e453087bc8fde7e561430)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:45:16.048071 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:45:16.048071 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Aug 09 18:45:16.048071 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: Aug 09 18:45:16.049276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:45:16.189348 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b1c7093d4e453087bc8fde7e561430', 'request_id': 'req-bd76409f-383a-46b3-abcd-c075d11693ca'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:45:16.203076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b1c7093d4e453087bc8fde7e561430', 'request_id': 'req-bd76409f-383a-46b3-abcd-c075d11693ca'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:45:16.209312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b1c7093d4e453087bc8fde7e561430', 'request_id': 'req-bd76409f-383a-46b3-abcd-c075d11693ca'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:45:16.324625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] The token cache is being invalidate because project 69b1c7093d4e453087bc8fde7e561430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:45:16.324914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:45:16.325457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bd76409f-383a-46b3-abcd-c075d11693ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:45:16.326785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 749/1510] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:45:15 2022] DELETE /identity/v3/projects/69b1c7093d4e453087bc8fde7e561430 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:45:25.382703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=62 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:45:26.322515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:46:45.042053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfc0a7-2e5e-48b8-934d-8262759cbc0f None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:45.042664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfc0a7-2e5e-48b8-934d-8262759cbc0f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:45.042664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfc0a7-2e5e-48b8-934d-8262759cbc0f None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:45.043947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:46:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:46:45.048983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1707e-e666-4618-b70f-576467931e81 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:45.049441 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1707e-e666-4618-b70f-576467931e81 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:45.049563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1707e-e666-4618-b70f-576467931e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:45.129909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7fe1707e-e666-4618-b70f-576467931e81 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:46:45.131256 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7fe1707e-e666-4618-b70f-576467931e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:46:45.241293 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:46:45.396322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfc0a7-2e5e-48b8-934d-8262759cbc0f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:46:45.398566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-91dfc0a7-2e5e-48b8-934d-8262759cbc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:46:45.521292 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfc0a7-2e5e-48b8-934d-8262759cbc0f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:46:45.527422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6010a-826b-4d3a-946c-00a6ab54d003 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:46:45.528019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6010a-826b-4d3a-946c-00a6ab54d003 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:45.528296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6010a-826b-4d3a-946c-00a6ab54d003 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:45.528563 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6010a-826b-4d3a-946c-00a6ab54d003 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:45.536462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6010a-826b-4d3a-946c-00a6ab54d003 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:46:45.538085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6010a-826b-4d3a-946c-00a6ab54d003 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:46:45.547371 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 763/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:46:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:46:46.389435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64dca2-457d-4da3-aede-24185b22c75d None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:46.390026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64dca2-457d-4da3-aede-24185b22c75d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:46.390026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64dca2-457d-4da3-aede-24185b22c75d None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:46.391417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:46:46.396262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7c21e-d5dc-4436-aff1-1004e3950450 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:46.396672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7c21e-d5dc-4436-aff1-1004e3950450 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:46.396811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7c21e-d5dc-4436-aff1-1004e3950450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:46.418652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-9fa7c21e-d5dc-4436-aff1-1004e3950450 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:46:46.419833 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9fa7c21e-d5dc-4436-aff1-1004e3950450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:46:46.428066 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:46:46.593624 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64dca2-457d-4da3-aede-24185b22c75d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:46:46.608793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64dca2-457d-4da3-aede-24185b22c75d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:46:46.614272 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb2d32-0646-4dbd-90ef-c892bf1c6a66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:46:46.614873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb2d32-0646-4dbd-90ef-c892bf1c6a66 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:46.615157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb2d32-0646-4dbd-90ef-c892bf1c6a66 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:46.615433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb2d32-0646-4dbd-90ef-c892bf1c6a66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:46.623787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb2d32-0646-4dbd-90ef-c892bf1c6a66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:46:46.625371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb2d32-0646-4dbd-90ef-c892bf1c6a66 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:46:46.634847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 752/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:46:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:46:47.111325 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252585ad-5b9a-425a-9e5f-f8bdb756a4bd None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:47.111607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252585ad-5b9a-425a-9e5f-f8bdb756a4bd None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:47.111879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252585ad-5b9a-425a-9e5f-f8bdb756a4bd None None] PATH_INFO: `/v3` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:47.113158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 765/1517] 10.222.0.45 () {54 vars in 925 bytes} [Tue Aug 9 18:46:47 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:46:47.119353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27db94e-4128-40d0-9220-50acbcd4bbd0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:47.119634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27db94e-4128-40d0-9220-50acbcd4bbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:47.119905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27db94e-4128-40d0-9220-50acbcd4bbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:47.179231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c27db94e-4128-40d0-9220-50acbcd4bbd0 None None] MFA Rules not processed for user `5e0aa6b215f646c280af245a89ccf849`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:46:47.180443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c27db94e-4128-40d0-9220-50acbcd4bbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:46:47.196902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:46:47.250474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 753/1518] 10.222.0.45 () {58 vars in 1015 bytes} [Tue Aug 9 18:46:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:46:51.354091 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f9546-e7ef-430a-a17a-a834c05c4be4 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:51.354657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f9546-e7ef-430a-a17a-a834c05c4be4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:51.354657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f9546-e7ef-430a-a17a-a834c05c4be4 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:51.355957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 766/1519] 10.222.0.57 () {54 vars in 961 bytes} [Tue Aug 9 18:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:46:51.360978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25fc33-090f-4c64-b058-af61a526ccdb None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:51.361411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25fc33-090f-4c64-b058-af61a526ccdb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:51.361505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25fc33-090f-4c64-b058-af61a526ccdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:51.382807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-cc25fc33-090f-4c64-b058-af61a526ccdb None None] MFA Rules not processed for user `5e0aa6b215f646c280af245a89ccf849`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:46:51.384056 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cc25fc33-090f-4c64-b058-af61a526ccdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:46:51.393395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 754/1520] 10.222.0.57 () {58 vars in 1058 bytes} [Tue Aug 9 18:46:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:46:51.401141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f9546-e7ef-430a-a17a-a834c05c4be4 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:46:51.414230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f9546-e7ef-430a-a17a-a834c05c4be4 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:46:51.420855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e5b74-2a68-479b-b509-fe344d589068 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0aa6b215f646c280af245a89ccf849', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:46:51.421503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e5b74-2a68-479b-b509-fe344d589068 service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:51.421895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e5b74-2a68-479b-b509-fe344d589068 service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:51.422208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e5b74-2a68-479b-b509-fe344d589068 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:51.431295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e5b74-2a68-479b-b509-fe344d589068 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:46:51.433397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e5b74-2a68-479b-b509-fe344d589068 service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:46:51.443618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 767/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:46:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:46:57.892763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4003a1-c5f7-4612-b485-73de1b1fb5f8 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:57.893366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4003a1-c5f7-4612-b485-73de1b1fb5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:57.893366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4003a1-c5f7-4612-b485-73de1b1fb5f8 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:57.894701 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:46:57.899701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4f945-c0c0-43f1-b36e-2dc728e12513 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:57.900182 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4f945-c0c0-43f1-b36e-2dc728e12513 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:57.900306 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4f945-c0c0-43f1-b36e-2dc728e12513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:57.919712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-3bd4f945-c0c0-43f1-b36e-2dc728e12513 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:46:57.920910 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3bd4f945-c0c0-43f1-b36e-2dc728e12513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:46:57.930026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:46:58.085514 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4003a1-c5f7-4612-b485-73de1b1fb5f8 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:46:58.100215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4003a1-c5f7-4612-b485-73de1b1fb5f8 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:46:58.106023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df34580-f2f9-447a-b0ee-8bcfff515eb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:46:58.106631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df34580-f2f9-447a-b0ee-8bcfff515eb5 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:58.106914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df34580-f2f9-447a-b0ee-8bcfff515eb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:58.107191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df34580-f2f9-447a-b0ee-8bcfff515eb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:58.116354 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df34580-f2f9-447a-b0ee-8bcfff515eb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:46:58.117974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df34580-f2f9-447a-b0ee-8bcfff515eb5 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:46:58.128339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 756/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:46:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:46:58.962980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac148d09-8ecc-4ff5-ac3e-ca3179a177b1 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:58.963564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac148d09-8ecc-4ff5-ac3e-ca3179a177b1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:58.963687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac148d09-8ecc-4ff5-ac3e-ca3179a177b1 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:58.966284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 18:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:46:58.971420 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42235e65-6d59-478c-a2f6-1813b9a26f7a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:58.971889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42235e65-6d59-478c-a2f6-1813b9a26f7a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:58.971985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42235e65-6d59-478c-a2f6-1813b9a26f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:59.002754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-42235e65-6d59-478c-a2f6-1813b9a26f7a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:46:59.006752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-42235e65-6d59-478c-a2f6-1813b9a26f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:46:59.014682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 18:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:46:59.165175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac148d09-8ecc-4ff5-ac3e-ca3179a177b1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:46:59.180167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac148d09-8ecc-4ff5-ac3e-ca3179a177b1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:46:59.186499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580b575-84cd-4a1b-a3bf-af34c4a250ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:46:59.187105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580b575-84cd-4a1b-a3bf-af34c4a250ba service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:46:59.187392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580b575-84cd-4a1b-a3bf-af34c4a250ba service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:46:59.187683 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580b575-84cd-4a1b-a3bf-af34c4a250ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:46:59.196039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580b575-84cd-4a1b-a3bf-af34c4a250ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:46:59.197656 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580b575-84cd-4a1b-a3bf-af34c4a250ba service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:46:59.207267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 770/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:46:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:47:03.275127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aa361-52b1-4c13-827b-90f6675df319 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:47:03.275566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aa361-52b1-4c13-827b-90f6675df319 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:47:03.275669 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aa361-52b1-4c13-827b-90f6675df319 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:47:03.277043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:47:03.282296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2179025-5e7f-405c-902b-f6c74864782d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:47:03.282763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2179025-5e7f-405c-902b-f6c74864782d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:47:03.282870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2179025-5e7f-405c-902b-f6c74864782d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:47:03.302263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d2179025-5e7f-405c-902b-f6c74864782d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:47:03.303456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d2179025-5e7f-405c-902b-f6c74864782d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:47:03.311832 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:47:03.393228 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526aa361-52b1-4c13-827b-90f6675df319 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:47:03.404623 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-526aa361-52b1-4c13-827b-90f6675df319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:47:03.475012 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526aa361-52b1-4c13-827b-90f6675df319 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:47:03.478765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cdd61-efe2-4726-afca-52c5e36bbd0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:47:03.479850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cdd61-efe2-4726-afca-52c5e36bbd0f service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:47:03.480135 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cdd61-efe2-4726-afca-52c5e36bbd0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:47:03.480405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cdd61-efe2-4726-afca-52c5e36bbd0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:47:03.489729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cdd61-efe2-4726-afca-52c5e36bbd0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:47:03.494099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cdd61-efe2-4726-afca-52c5e36bbd0f service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:47:03.504238 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 759/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:47:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:47:05.523404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787b027-f50e-4454-a5dc-a23028f5e565 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:47:05.523926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787b027-f50e-4454-a5dc-a23028f5e565 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:47:05.523926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787b027-f50e-4454-a5dc-a23028f5e565 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:47:05.525155 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:47:05.531209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29871a74-e9a9-4757-aca4-2e538a55fb17 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:47:05.531672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29871a74-e9a9-4757-aca4-2e538a55fb17 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:47:05.531792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29871a74-e9a9-4757-aca4-2e538a55fb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:47:05.552828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-29871a74-e9a9-4757-aca4-2e538a55fb17 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:47:05.554072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-29871a74-e9a9-4757-aca4-2e538a55fb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:47:05.563754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:47:05.602065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787b027-f50e-4454-a5dc-a23028f5e565 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:47:05.608270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=75 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:47:05.614082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787b027-f50e-4454-a5dc-a23028f5e565 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:47:05.617753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d074165-31df-43d0-bdc6-e27da1198e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:47:05.618344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d074165-31df-43d0-bdc6-e27da1198e8d service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:47:05.618881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d074165-31df-43d0-bdc6-e27da1198e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:47:05.618881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d074165-31df-43d0-bdc6-e27da1198e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:47:05.628430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d074165-31df-43d0-bdc6-e27da1198e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:47:05.629906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d074165-31df-43d0-bdc6-e27da1198e8d service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:47:05.639725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 773/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:47:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:47:09.238729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba53c2-d8c1-438f-9107-303653e4624a None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:47:09.239403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba53c2-d8c1-438f-9107-303653e4624a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:47:09.239403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba53c2-d8c1-438f-9107-303653e4624a None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:47:09.240656 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:47:09.246517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7d3c7-797e-4235-b3ca-e0af755100f6 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:47:09.246952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7d3c7-797e-4235-b3ca-e0af755100f6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:47:09.247057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7d3c7-797e-4235-b3ca-e0af755100f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:47:09.266941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-49b7d3c7-797e-4235-b3ca-e0af755100f6 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:47:09.268146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-49b7d3c7-797e-4235-b3ca-e0af755100f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:47:09.276173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:47:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:47:09.591308 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba53c2-d8c1-438f-9107-303653e4624a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:47:09.592985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c4ba53c2-d8c1-438f-9107-303653e4624a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:47:09.660855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:47:09.674524 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba53c2-d8c1-438f-9107-303653e4624a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:47:09.678667 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd91198-2123-4372-8499-477dbe8353d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:47:09.679270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd91198-2123-4372-8499-477dbe8353d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:47:09.679550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd91198-2123-4372-8499-477dbe8353d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:47:09.679823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd91198-2123-4372-8499-477dbe8353d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:47:09.697500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd91198-2123-4372-8499-477dbe8353d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:47:09.699098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd91198-2123-4372-8499-477dbe8353d8 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:47:09.710760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 762/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:47:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:47:11.736929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427ca11-fa4a-4313-9632-bbe3f4944bd6 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:47:11.737551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427ca11-fa4a-4313-9632-bbe3f4944bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:47:11.737551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427ca11-fa4a-4313-9632-bbe3f4944bd6 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:47:11.738889 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 18:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:47:11.744135 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2d2c2-ceb4-4e90-b9a6-bb005d07b118 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:47:11.745172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2d2c2-ceb4-4e90-b9a6-bb005d07b118 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:47:11.745404 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2d2c2-ceb4-4e90-b9a6-bb005d07b118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:47:11.765440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5ab2d2c2-ceb4-4e90-b9a6-bb005d07b118 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:47:11.766553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5ab2d2c2-ceb4-4e90-b9a6-bb005d07b118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:47:11.774711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 18:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:47:12.080193 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427ca11-fa4a-4313-9632-bbe3f4944bd6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:47:12.081912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a427ca11-fa4a-4313-9632-bbe3f4944bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:47:12.095698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427ca11-fa4a-4313-9632-bbe3f4944bd6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:47:12.099248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc8a66-fa18-4bdd-b836-5e5d9c344c04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:47:12.100108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc8a66-fa18-4bdd-b836-5e5d9c344c04 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:47:12.100397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc8a66-fa18-4bdd-b836-5e5d9c344c04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:47:12.100710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc8a66-fa18-4bdd-b836-5e5d9c344c04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:47:12.109893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc8a66-fa18-4bdd-b836-5e5d9c344c04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:47:12.111316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc8a66-fa18-4bdd-b836-5e5d9c344c04 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:47:12.121131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 776/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:47:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:47:21.759871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=11 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:47:22.116537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=14 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:36.315813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c37df-9f0d-434b-b00c-3756c1096f6d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.316509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c37df-9f0d-434b-b00c-3756c1096f6d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.316509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c37df-9f0d-434b-b00c-3756c1096f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.340539 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a54c37df-9f0d-434b-b00c-3756c1096f6d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.341782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a54c37df-9f0d-434b-b00c-3756c1096f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.353620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 764/1540] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.362684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c6f1-639b-412d-8fa2-a09b754fa602 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.363825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c6f1-639b-412d-8fa2-a09b754fa602 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.363825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c6f1-639b-412d-8fa2-a09b754fa602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.402617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0e01c6f1-639b-412d-8fa2-a09b754fa602 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.404571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0e01c6f1-639b-412d-8fa2-a09b754fa602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.417453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 777/1541] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.422538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c37df-9f0d-434b-b00c-3756c1096f6d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:36.438664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c37df-9f0d-434b-b00c-3756c1096f6d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:36.443653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0786e4-05db-4b0b-929e-c22b51508672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:36.444287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0786e4-05db-4b0b-929e-c22b51508672 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.444572 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0786e4-05db-4b0b-929e-c22b51508672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.444911 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0786e4-05db-4b0b-929e-c22b51508672 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.445608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0786e4-05db-4b0b-929e-c22b51508672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:48:36.449199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0786e4-05db-4b0b-929e-c22b51508672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:36.451386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0786e4-05db-4b0b-929e-c22b51508672 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:36.470779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 765/1542] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:48:36.480015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01c6f1-639b-412d-8fa2-a09b754fa602 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:36.497990 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01c6f1-639b-412d-8fa2-a09b754fa602 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:36.506133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c1b7f-2772-49ea-862d-17d447594f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:36.506757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c1b7f-2772-49ea-862d-17d447594f0d admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.507039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c1b7f-2772-49ea-862d-17d447594f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.507563 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c1b7f-2772-49ea-862d-17d447594f0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.508734 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c1b7f-2772-49ea-862d-17d447594f0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:36.510685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c1b7f-2772-49ea-862d-17d447594f0d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:36.540942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1968e22-5371-4d6a-aaf8-d08375d4a640 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.541145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1968e22-5371-4d6a-aaf8-d08375d4a640 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.541481 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1968e22-5371-4d6a-aaf8-d08375d4a640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.579247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b1968e22-5371-4d6a-aaf8-d08375d4a640 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.580500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b1968e22-5371-4d6a-aaf8-d08375d4a640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.591429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 766/1543] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.592867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053f045-a337-4ac6-9865-2965670a1f48 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.593192 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053f045-a337-4ac6-9865-2965670a1f48 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.593470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053f045-a337-4ac6-9865-2965670a1f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.614530 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b053f045-a337-4ac6-9865-2965670a1f48 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.615729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b053f045-a337-4ac6-9865-2965670a1f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.623620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 767/1544] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.624763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c5e7c-cbba-46e5-ad79-ddbe6d9865c8 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.625046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c5e7c-cbba-46e5-ad79-ddbe6d9865c8 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.625318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c5e7c-cbba-46e5-ad79-ddbe6d9865c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.643020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 778/1545] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:36.645043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-461c5e7c-cbba-46e5-ad79-ddbe6d9865c8 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.645201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e46b55-19ef-46b0-b9f1-f0d8c8d51ae2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.645419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e46b55-19ef-46b0-b9f1-f0d8c8d51ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.645746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e46b55-19ef-46b0-b9f1-f0d8c8d51ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.650760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-461c5e7c-cbba-46e5-ad79-ddbe6d9865c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.666573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-25e46b55-19ef-46b0-b9f1-f0d8c8d51ae2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.667790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-25e46b55-19ef-46b0-b9f1-f0d8c8d51ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.711556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 768/1546] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.715469 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42f78b-e96e-43f8-b23d-b916d4acd310 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.719626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42f78b-e96e-43f8-b23d-b916d4acd310 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.719626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42f78b-e96e-43f8-b23d-b916d4acd310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.729932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 779/1547] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.731463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975efd73-67e7-4c4f-8bfc-f55ed777d6cc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.731733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975efd73-67e7-4c4f-8bfc-f55ed777d6cc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.732003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975efd73-67e7-4c4f-8bfc-f55ed777d6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.741803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3c42f78b-e96e-43f8-b23d-b916d4acd310 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.743031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3c42f78b-e96e-43f8-b23d-b916d4acd310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.751023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 769/1548] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.752135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-975efd73-67e7-4c4f-8bfc-f55ed777d6cc None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.752263 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575f493-a4ef-4eba-9f7a-3aedb0375718 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.752483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575f493-a4ef-4eba-9f7a-3aedb0375718 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.752760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575f493-a4ef-4eba-9f7a-3aedb0375718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.753309 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-975efd73-67e7-4c4f-8bfc-f55ed777d6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.762158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 780/1549] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.766929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377c861-fb35-4d24-8595-135150794394 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.767273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377c861-fb35-4d24-8595-135150794394 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.767553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377c861-fb35-4d24-8595-135150794394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.773883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0575f493-a4ef-4eba-9f7a-3aedb0375718 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.775089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0575f493-a4ef-4eba-9f7a-3aedb0375718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.782993 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 770/1550] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.784306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea98a4b-4c4a-48d0-a452-f231083118eb None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.784587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea98a4b-4c4a-48d0-a452-f231083118eb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.785418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea98a4b-4c4a-48d0-a452-f231083118eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.788454 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d377c861-fb35-4d24-8595-135150794394 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.794201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d377c861-fb35-4d24-8595-135150794394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.799040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 781/1551] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.799945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377c861-fb35-4d24-8595-135150794394 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:36.805355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-dea98a4b-4c4a-48d0-a452-f231083118eb None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.807340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dea98a4b-4c4a-48d0-a452-f231083118eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.809882 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377c861-fb35-4d24-8595-135150794394 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:36.814603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0d43a-6f5c-4547-9b45-60ab74e0ea66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:36.815208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0d43a-6f5c-4547-9b45-60ab74e0ea66 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.815490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0d43a-6f5c-4547-9b45-60ab74e0ea66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.815609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 771/1552] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.815777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0d43a-6f5c-4547-9b45-60ab74e0ea66 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.816410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0d43a-6f5c-4547-9b45-60ab74e0ea66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:36.816932 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c9e18-4388-4498-a8a7-195855e2c9ca None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.817222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c9e18-4388-4498-a8a7-195855e2c9ca None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.817431 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-3ca0d43a-6f5c-4547-9b45-60ab74e0ea66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:48:36.817431 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Aug 09 18:48:36.817431 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: Aug 09 18:48:36.817728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c9e18-4388-4498-a8a7-195855e2c9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.818830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0d43a-6f5c-4547-9b45-60ab74e0ea66 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:36.837265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1e4c9e18-4388-4498-a8a7-195855e2c9ca None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.838513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1e4c9e18-4388-4498-a8a7-195855e2c9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.846460 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 772/1553] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.848197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e79c3-4008-4c11-b00c-74085579b582 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.848481 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e79c3-4008-4c11-b00c-74085579b582 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.848764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e79c3-4008-4c11-b00c-74085579b582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.867571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-df0e79c3-4008-4c11-b00c-74085579b582 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.871069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-df0e79c3-4008-4c11-b00c-74085579b582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.876700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 773/1554] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.877873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682ca5d-cd68-41bb-a988-370ccbb44d33 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.878159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682ca5d-cd68-41bb-a988-370ccbb44d33 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.878441 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682ca5d-cd68-41bb-a988-370ccbb44d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.902491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 782/1555] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:36.902710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1682ca5d-cd68-41bb-a988-370ccbb44d33 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.902834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737f43c-e6dd-47a5-958b-152d37155b99 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.902938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1682ca5d-cd68-41bb-a988-370ccbb44d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.903034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737f43c-e6dd-47a5-958b-152d37155b99 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.903034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737f43c-e6dd-47a5-958b-152d37155b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.909308 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 774/1556] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.910448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682ca5d-cd68-41bb-a988-370ccbb44d33 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:36.919866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682ca5d-cd68-41bb-a988-370ccbb44d33 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:36.920955 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6737f43c-e6dd-47a5-958b-152d37155b99 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.922179 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6737f43c-e6dd-47a5-958b-152d37155b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.924440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f766a3-2488-46fb-b762-f27dbd352597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:36.925029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f766a3-2488-46fb-b762-f27dbd352597 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.925309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f766a3-2488-46fb-b762-f27dbd352597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.925904 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f766a3-2488-46fb-b762-f27dbd352597 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.926566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f766a3-2488-46fb-b762-f27dbd352597 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:48:36.926875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f766a3-2488-46fb-b762-f27dbd352597 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:36.928485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f766a3-2488-46fb-b762-f27dbd352597 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:36.930654 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 783/1557] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.931922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3ed53-d69f-43c4-8de7-032b467c10de None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.932209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3ed53-d69f-43c4-8de7-032b467c10de None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.932484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3ed53-d69f-43c4-8de7-032b467c10de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.942257 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 775/1558] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:48:36.942908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f766a3-2488-46fb-b762-f27dbd352597 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:36.952844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f766a3-2488-46fb-b762-f27dbd352597 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:36.953355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-51f3ed53-d69f-43c4-8de7-032b467c10de None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:36.954651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-51f3ed53-d69f-43c4-8de7-032b467c10de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:36.958310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fc162-1cfe-4629-8152-1a0c628a0fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:36.970896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fc162-1cfe-4629-8152-1a0c628a0fab admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.970896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fc162-1cfe-4629-8152-1a0c628a0fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.970896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fc162-1cfe-4629-8152-1a0c628a0fab admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.970896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fc162-1cfe-4629-8152-1a0c628a0fab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:48:36.970896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fc162-1cfe-4629-8152-1a0c628a0fab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:36.970896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fc162-1cfe-4629-8152-1a0c628a0fab admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:36.971458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 784/1559] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:36.971458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3ed53-d69f-43c4-8de7-032b467c10de None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:36.974828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 776/1560] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:48:36.975934 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fc162-1cfe-4629-8152-1a0c628a0fab admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:36.976674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3ed53-d69f-43c4-8de7-032b467c10de None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:36.981604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1a6cf-0d0b-4bcd-9914-ed67c5db547e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:36.982208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1a6cf-0d0b-4bcd-9914-ed67c5db547e admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.982501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1a6cf-0d0b-4bcd-9914-ed67c5db547e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.982750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1a6cf-0d0b-4bcd-9914-ed67c5db547e admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.983400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1a6cf-0d0b-4bcd-9914-ed67c5db547e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:48:36.983702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1a6cf-0d0b-4bcd-9914-ed67c5db547e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:36.984706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-e2a1a6cf-0d0b-4bcd-9914-ed67c5db547e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:48:36.984706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Aug 09 18:48:36.984706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: Aug 09 18:48:36.985756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1a6cf-0d0b-4bcd-9914-ed67c5db547e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:36.987137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fc162-1cfe-4629-8152-1a0c628a0fab admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:36.991677 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ec0c5-c770-4cd3-a09b-e5131a2e9cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:36.992263 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ec0c5-c770-4cd3-a09b-e5131a2e9cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:36.992974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ec0c5-c770-4cd3-a09b-e5131a2e9cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:36.992974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ec0c5-c770-4cd3-a09b-e5131a2e9cb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:36.993475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ec0c5-c770-4cd3-a09b-e5131a2e9cb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:48:36.993790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ec0c5-c770-4cd3-a09b-e5131a2e9cb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:36.995361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ec0c5-c770-4cd3-a09b-e5131a2e9cb8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.005816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 785/1561] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:48:37.005816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1a6cf-0d0b-4bcd-9914-ed67c5db547e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.009319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1a6cf-0d0b-4bcd-9914-ed67c5db547e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.011624 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 777/1562] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:48:37.012680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ec0c5-c770-4cd3-a09b-e5131a2e9cb8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.014225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c60fc8-81fd-4af3-984e-951a0cf8a95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.014972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c60fc8-81fd-4af3-984e-951a0cf8a95b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.015261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c60fc8-81fd-4af3-984e-951a0cf8a95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.015550 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c60fc8-81fd-4af3-984e-951a0cf8a95b admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.016181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c60fc8-81fd-4af3-984e-951a0cf8a95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.017810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c60fc8-81fd-4af3-984e-951a0cf8a95b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.024281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ec0c5-c770-4cd3-a09b-e5131a2e9cb8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.028943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d40e96-a722-43f9-a1a5-cee6105f3b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.029557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d40e96-a722-43f9-a1a5-cee6105f3b2c admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.037585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d40e96-a722-43f9-a1a5-cee6105f3b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.037585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d40e96-a722-43f9-a1a5-cee6105f3b2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.037787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 786/1563] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:37.037869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d40e96-a722-43f9-a1a5-cee6105f3b2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:48:37.037952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c60fc8-81fd-4af3-984e-951a0cf8a95b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.038039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d40e96-a722-43f9-a1a5-cee6105f3b2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.038039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d40e96-a722-43f9-a1a5-cee6105f3b2c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.041467 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c60fc8-81fd-4af3-984e-951a0cf8a95b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.045642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 778/1564] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:48:37.046095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc5dde-4d30-49a2-ac15-495439fff9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.046391 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d40e96-a722-43f9-a1a5-cee6105f3b2c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.046750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc5dde-4d30-49a2-ac15-495439fff9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.047036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc5dde-4d30-49a2-ac15-495439fff9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.047362 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc5dde-4d30-49a2-ac15-495439fff9b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.048019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc5dde-4d30-49a2-ac15-495439fff9b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:48:37.048321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc5dde-4d30-49a2-ac15-495439fff9b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.049963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc5dde-4d30-49a2-ac15-495439fff9b1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.056899 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d40e96-a722-43f9-a1a5-cee6105f3b2c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.061864 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109185e7-04c5-4959-9546-1e4c78102fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.062459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109185e7-04c5-4959-9546-1e4c78102fbd admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.062737 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109185e7-04c5-4959-9546-1e4c78102fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.063005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109185e7-04c5-4959-9546-1e4c78102fbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.063657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109185e7-04c5-4959-9546-1e4c78102fbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.064255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 787/1565] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:48:37.064690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-109185e7-04c5-4959-9546-1e4c78102fbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:48:37.064690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Aug 09 18:48:37.064690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: Aug 09 18:48:37.065100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc5dde-4d30-49a2-ac15-495439fff9b1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.065997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109185e7-04c5-4959-9546-1e4c78102fbd admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.075814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc5dde-4d30-49a2-ac15-495439fff9b1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.080493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2629344-8147-4239-a137-0e40b5f3a8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.081082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2629344-8147-4239-a137-0e40b5f3a8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.081391 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2629344-8147-4239-a137-0e40b5f3a8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.082385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2629344-8147-4239-a137-0e40b5f3a8b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.083036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2629344-8147-4239-a137-0e40b5f3a8b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:48:37.083342 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2629344-8147-4239-a137-0e40b5f3a8b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.084937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2629344-8147-4239-a137-0e40b5f3a8b2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.096930 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 788/1566] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:48:37.097758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2629344-8147-4239-a137-0e40b5f3a8b2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.107416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2629344-8147-4239-a137-0e40b5f3a8b2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.111981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849de20d-ce2c-45b9-902e-81ffdce9d555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.112544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849de20d-ce2c-45b9-902e-81ffdce9d555 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.112823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849de20d-ce2c-45b9-902e-81ffdce9d555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.113120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849de20d-ce2c-45b9-902e-81ffdce9d555 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.113760 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849de20d-ce2c-45b9-902e-81ffdce9d555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.115661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849de20d-ce2c-45b9-902e-81ffdce9d555 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.185815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 779/1567] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:37 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:37.186630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109185e7-04c5-4959-9546-1e4c78102fbd admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.200221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:37.221667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 789/1568] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:37 2022] POST /identity/v3/projects => generated 357 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:37.221667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849de20d-ce2c-45b9-902e-81ffdce9d555 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.231602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:37.249124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109185e7-04c5-4959-9546-1e4c78102fbd admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.285136 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849de20d-ce2c-45b9-902e-81ffdce9d555 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.290115 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b8f73-0ac3-4f98-80e5-79fa5f26d150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.290755 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b8f73-0ac3-4f98-80e5-79fa5f26d150 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.291041 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b8f73-0ac3-4f98-80e5-79fa5f26d150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.291311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b8f73-0ac3-4f98-80e5-79fa5f26d150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.291939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b8f73-0ac3-4f98-80e5-79fa5f26d150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.293959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b8f73-0ac3-4f98-80e5-79fa5f26d150 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.314635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874fc74-9279-45f2-a475-3f3844b55a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.318172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874fc74-9279-45f2-a475-3f3844b55a3f admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.318172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874fc74-9279-45f2-a475-3f3844b55a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.318172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874fc74-9279-45f2-a475-3f3844b55a3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.318172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874fc74-9279-45f2-a475-3f3844b55a3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.319106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874fc74-9279-45f2-a475-3f3844b55a3f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.380525 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 790/1569] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:37 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:37.381483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b8f73-0ac3-4f98-80e5-79fa5f26d150 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.414400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 780/1570] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:37 2022] POST /identity/v3/projects => generated 391 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:37.415419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874fc74-9279-45f2-a475-3f3844b55a3f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.443366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b8f73-0ac3-4f98-80e5-79fa5f26d150 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.480205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874fc74-9279-45f2-a475-3f3844b55a3f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.484994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fb4dd-a131-4f5a-92d3-409a76f566f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.485632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fb4dd-a131-4f5a-92d3-409a76f566f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.485916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fb4dd-a131-4f5a-92d3-409a76f566f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.486194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fb4dd-a131-4f5a-92d3-409a76f566f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.486817 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fb4dd-a131-4f5a-92d3-409a76f566f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.489232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fb4dd-a131-4f5a-92d3-409a76f566f2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.498747 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1a0e6-af49-43f0-8283-6e328e8b8680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.499360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1a0e6-af49-43f0-8283-6e328e8b8680 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.499638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1a0e6-af49-43f0-8283-6e328e8b8680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.499907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1a0e6-af49-43f0-8283-6e328e8b8680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.500540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1a0e6-af49-43f0-8283-6e328e8b8680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.502166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1a0e6-af49-43f0-8283-6e328e8b8680 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.512649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 791/1571] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:37.513435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1a0e6-af49-43f0-8283-6e328e8b8680 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.524317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1a0e6-af49-43f0-8283-6e328e8b8680 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.529009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a81a1-b159-4ab3-8170-a10469325bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.529634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a81a1-b159-4ab3-8170-a10469325bce admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.529917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a81a1-b159-4ab3-8170-a10469325bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.530191 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a81a1-b159-4ab3-8170-a10469325bce admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.530903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a81a1-b159-4ab3-8170-a10469325bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.532753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a81a1-b159-4ab3-8170-a10469325bce admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.578957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 781/1572] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:37 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:37.578957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fb4dd-a131-4f5a-92d3-409a76f566f2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.609714 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:37.642123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fb4dd-a131-4f5a-92d3-409a76f566f2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.646871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf550b4-e5b7-404a-af94-a91d22b6dc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.647475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf550b4-e5b7-404a-af94-a91d22b6dc44 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.647758 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf550b4-e5b7-404a-af94-a91d22b6dc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.648030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf550b4-e5b7-404a-af94-a91d22b6dc44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.648658 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf550b4-e5b7-404a-af94-a91d22b6dc44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.650625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf550b4-e5b7-404a-af94-a91d22b6dc44 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.689011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 792/1573] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:37 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:37.691551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a81a1-b159-4ab3-8170-a10469325bce admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.703593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:37.751893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a81a1-b159-4ab3-8170-a10469325bce admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.756803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef9683-c0c1-45b7-9abe-28aac72c39df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.757061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 782/1574] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:37 2022] POST /identity/v3/projects => generated 367 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:37.757415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef9683-c0c1-45b7-9abe-28aac72c39df admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.757802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef9683-c0c1-45b7-9abe-28aac72c39df admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.757929 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf550b4-e5b7-404a-af94-a91d22b6dc44 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.758087 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef9683-c0c1-45b7-9abe-28aac72c39df admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.758709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef9683-c0c1-45b7-9abe-28aac72c39df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.760866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef9683-c0c1-45b7-9abe-28aac72c39df admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.822846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf550b4-e5b7-404a-af94-a91d22b6dc44 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.827507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc748e-5590-4ebb-b792-7a359aa88aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.828113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc748e-5590-4ebb-b792-7a359aa88aea admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.828397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc748e-5590-4ebb-b792-7a359aa88aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.828676 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc748e-5590-4ebb-b792-7a359aa88aea admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.829376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc748e-5590-4ebb-b792-7a359aa88aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.831290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc748e-5590-4ebb-b792-7a359aa88aea admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.845401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 793/1575] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:37 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:37.845401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef9683-c0c1-45b7-9abe-28aac72c39df admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.853417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef9683-c0c1-45b7-9abe-28aac72c39df admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.857968 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead3d96-9c70-4051-90dc-64a5da20f77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.858537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead3d96-9c70-4051-90dc-64a5da20f77c admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.858814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead3d96-9c70-4051-90dc-64a5da20f77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.859084 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead3d96-9c70-4051-90dc-64a5da20f77c admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.859681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead3d96-9c70-4051-90dc-64a5da20f77c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.861613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead3d96-9c70-4051-90dc-64a5da20f77c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.925326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 783/1576] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:37 2022] POST /identity/v3/users => generated 279 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:37.925326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc748e-5590-4ebb-b792-7a359aa88aea admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.933841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc748e-5590-4ebb-b792-7a359aa88aea admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.940124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea1ca4-b7c5-40ac-9535-378a8ec52493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.940753 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea1ca4-b7c5-40ac-9535-378a8ec52493 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.941014 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea1ca4-b7c5-40ac-9535-378a8ec52493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.941298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 794/1577] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:37 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:37.941469 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea1ca4-b7c5-40ac-9535-378a8ec52493 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.941967 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea1ca4-b7c5-40ac-9535-378a8ec52493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.942784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead3d96-9c70-4051-90dc-64a5da20f77c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:37.943899 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea1ca4-b7c5-40ac-9535-378a8ec52493 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:37.954572 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead3d96-9c70-4051-90dc-64a5da20f77c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:37.959372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2431d23-5686-4a78-bc2a-18e85c12805c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:37.960089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2431d23-5686-4a78-bc2a-18e85c12805c admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:37.962116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2431d23-5686-4a78-bc2a-18e85c12805c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:37.962403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2431d23-5686-4a78-bc2a-18e85c12805c admin admin] PATH_INFO: `/v3/projects/539b956efa1b47799514fbdcac7e250c/users/7fde48fb289b4c7591658e27ce5ff55e/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:37.978595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2431d23-5686-4a78-bc2a-18e85c12805c admin admin] RBAC: Authorizing `identity:create_grant(project_id=539b956efa1b47799514fbdcac7e250c, user_id=7fde48fb289b4c7591658e27ce5ff55e, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:37.981274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2431d23-5686-4a78-bc2a-18e85c12805c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.030710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 784/1578] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:37 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:38.031526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea1ca4-b7c5-40ac-9535-378a8ec52493 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.041909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea1ca4-b7c5-40ac-9535-378a8ec52493 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.046509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca88ca-56fd-4a83-9231-8876828b07e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.047116 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca88ca-56fd-4a83-9231-8876828b07e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.047392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca88ca-56fd-4a83-9231-8876828b07e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.047664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca88ca-56fd-4a83-9231-8876828b07e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.048287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca88ca-56fd-4a83-9231-8876828b07e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.050233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca88ca-56fd-4a83-9231-8876828b07e7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.059905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 795/1579] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:37 2022] PUT /identity/v3/projects/539b956efa1b47799514fbdcac7e250c/users/7fde48fb289b4c7591658e27ce5ff55e/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:38.060716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2431d23-5686-4a78-bc2a-18e85c12805c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.124669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2431d23-5686-4a78-bc2a-18e85c12805c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.129155 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a912fe-fc60-448f-b25c-7b6c31790cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.129784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a912fe-fc60-448f-b25c-7b6c31790cbb admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.130064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a912fe-fc60-448f-b25c-7b6c31790cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.130331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a912fe-fc60-448f-b25c-7b6c31790cbb admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.130945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a912fe-fc60-448f-b25c-7b6c31790cbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.132981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a912fe-fc60-448f-b25c-7b6c31790cbb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.154207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 785/1580] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:38 2022] POST /identity/v3/users => generated 283 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:38.155519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca88ca-56fd-4a83-9231-8876828b07e7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.172078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca88ca-56fd-4a83-9231-8876828b07e7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.177095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3666f4-b9be-403a-ad8b-d6e82305fccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.177792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3666f4-b9be-403a-ad8b-d6e82305fccf admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.178069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3666f4-b9be-403a-ad8b-d6e82305fccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.178346 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3666f4-b9be-403a-ad8b-d6e82305fccf admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.178993 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3666f4-b9be-403a-ad8b-d6e82305fccf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.180916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3666f4-b9be-403a-ad8b-d6e82305fccf admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.236142 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 796/1581] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:38 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:38.237166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a912fe-fc60-448f-b25c-7b6c31790cbb admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.246809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a912fe-fc60-448f-b25c-7b6c31790cbb admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.251367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a68a5-e491-48cd-854e-d9239da54eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.251967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a68a5-e491-48cd-854e-d9239da54eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.252246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a68a5-e491-48cd-854e-d9239da54eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.252518 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a68a5-e491-48cd-854e-d9239da54eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.253202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a68a5-e491-48cd-854e-d9239da54eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.254863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a68a5-e491-48cd-854e-d9239da54eb4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.262174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 786/1582] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:38 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:38.262174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3666f4-b9be-403a-ad8b-d6e82305fccf admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.271201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 797/1583] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.271201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a68a5-e491-48cd-854e-d9239da54eb4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.274271 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3666f4-b9be-403a-ad8b-d6e82305fccf admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.279036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323f5d5-c9f4-4345-9e5b-fbca64193e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.279503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a68a5-e491-48cd-854e-d9239da54eb4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.279644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323f5d5-c9f4-4345-9e5b-fbca64193e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.279984 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323f5d5-c9f4-4345-9e5b-fbca64193e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.280294 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323f5d5-c9f4-4345-9e5b-fbca64193e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.280913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323f5d5-c9f4-4345-9e5b-fbca64193e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.282589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323f5d5-c9f4-4345-9e5b-fbca64193e0d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.284078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfc7c7-aa50-4e40-a676-21f149d47b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.284660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfc7c7-aa50-4e40-a676-21f149d47b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.285553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfc7c7-aa50-4e40-a676-21f149d47b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.287404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfc7c7-aa50-4e40-a676-21f149d47b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.287404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfc7c7-aa50-4e40-a676-21f149d47b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.288086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfc7c7-aa50-4e40-a676-21f149d47b0c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.296979 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 787/1584] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.296979 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323f5d5-c9f4-4345-9e5b-fbca64193e0d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.301866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 798/1585] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.301866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8c4ba-c8df-45b8-81c0-67168dd2360a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.301866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8c4ba-c8df-45b8-81c0-67168dd2360a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.301866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8c4ba-c8df-45b8-81c0-67168dd2360a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.306222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323f5d5-c9f4-4345-9e5b-fbca64193e0d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.310995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f1e58-00bb-4713-b903-c35b4aaad2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.311716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f1e58-00bb-4713-b903-c35b4aaad2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.312099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f1e58-00bb-4713-b903-c35b4aaad2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.312474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f1e58-00bb-4713-b903-c35b4aaad2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.313217 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f1e58-00bb-4713-b903-c35b4aaad2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.314996 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f1e58-00bb-4713-b903-c35b4aaad2d2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.319237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7ff8c4ba-c8df-45b8-81c0-67168dd2360a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:38.320421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7ff8c4ba-c8df-45b8-81c0-67168dd2360a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:38.325508 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 788/1586] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.326398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f1e58-00bb-4713-b903-c35b4aaad2d2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.330847 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 799/1587] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:38.332220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8c4ba-c8df-45b8-81c0-67168dd2360a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.336833 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f1e58-00bb-4713-b903-c35b4aaad2d2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.341589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5387e8-696e-42c9-a00e-c555d1dbf071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.342191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5387e8-696e-42c9-a00e-c555d1dbf071 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.342476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5387e8-696e-42c9-a00e-c555d1dbf071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.342750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5387e8-696e-42c9-a00e-c555d1dbf071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.343014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8c4ba-c8df-45b8-81c0-67168dd2360a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.343384 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5387e8-696e-42c9-a00e-c555d1dbf071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.344979 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5387e8-696e-42c9-a00e-c555d1dbf071 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.348253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9548aac-f91a-4a04-aaca-4333b38ac9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.348846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9548aac-f91a-4a04-aaca-4333b38ac9bd admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.349130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9548aac-f91a-4a04-aaca-4333b38ac9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.349385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9548aac-f91a-4a04-aaca-4333b38ac9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.350062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9548aac-f91a-4a04-aaca-4333b38ac9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.351644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9548aac-f91a-4a04-aaca-4333b38ac9bd admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.361424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 789/1588] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.361424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5387e8-696e-42c9-a00e-c555d1dbf071 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.362514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 800/1589] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.363590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9548aac-f91a-4a04-aaca-4333b38ac9bd admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.369903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5387e8-696e-42c9-a00e-c555d1dbf071 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.373177 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9548aac-f91a-4a04-aaca-4333b38ac9bd admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.374578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade438e-cb42-40c4-b64e-98143d674b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.375232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade438e-cb42-40c4-b64e-98143d674b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.375456 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade438e-cb42-40c4-b64e-98143d674b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.375736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade438e-cb42-40c4-b64e-98143d674b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.376371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade438e-cb42-40c4-b64e-98143d674b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.377842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb5731-2718-4121-a579-0652af38677d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.378009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade438e-cb42-40c4-b64e-98143d674b18 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.378416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb5731-2718-4121-a579-0652af38677d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.379215 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb5731-2718-4121-a579-0652af38677d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.379502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb5731-2718-4121-a579-0652af38677d admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.380108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb5731-2718-4121-a579-0652af38677d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.381694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb5731-2718-4121-a579-0652af38677d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.387024 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:38.388528 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 790/1590] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.389328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade438e-cb42-40c4-b64e-98143d674b18 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.393799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 801/1591] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.398397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb5731-2718-4121-a579-0652af38677d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.399671 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade438e-cb42-40c4-b64e-98143d674b18 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.404486 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a614aa-84e8-4e10-b9c4-97e08518851c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.404691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb5731-2718-4121-a579-0652af38677d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.405080 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a614aa-84e8-4e10-b9c4-97e08518851c admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.405366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a614aa-84e8-4e10-b9c4-97e08518851c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.405677 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a614aa-84e8-4e10-b9c4-97e08518851c admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.406364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a614aa-84e8-4e10-b9c4-97e08518851c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.407948 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a614aa-84e8-4e10-b9c4-97e08518851c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.409287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c42c6f-4e5b-48a9-ac8e-1a089b0dbbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.409947 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c42c6f-4e5b-48a9-ac8e-1a089b0dbbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.410224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c42c6f-4e5b-48a9-ac8e-1a089b0dbbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.410338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:38.410567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c42c6f-4e5b-48a9-ac8e-1a089b0dbbe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.411196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c42c6f-4e5b-48a9-ac8e-1a089b0dbbe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.412836 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c42c6f-4e5b-48a9-ac8e-1a089b0dbbe7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.422195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 791/1592] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.422195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a614aa-84e8-4e10-b9c4-97e08518851c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.423545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 802/1593] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.423545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c42c6f-4e5b-48a9-ac8e-1a089b0dbbe7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.433697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a614aa-84e8-4e10-b9c4-97e08518851c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.438387 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed60a46-44e8-4ab3-b097-8a8e05d26b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.439062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed60a46-44e8-4ab3-b097-8a8e05d26b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.439482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed60a46-44e8-4ab3-b097-8a8e05d26b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.439632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed60a46-44e8-4ab3-b097-8a8e05d26b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.440281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed60a46-44e8-4ab3-b097-8a8e05d26b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.442459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed60a46-44e8-4ab3-b097-8a8e05d26b72 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.452801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 792/1594] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.453731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed60a46-44e8-4ab3-b097-8a8e05d26b72 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.463951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed60a46-44e8-4ab3-b097-8a8e05d26b72 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.468680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0c793-c378-4cf1-99c1-f2e72e9468dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.469265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0c793-c378-4cf1-99c1-f2e72e9468dd admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.469575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0c793-c378-4cf1-99c1-f2e72e9468dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.469856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0c793-c378-4cf1-99c1-f2e72e9468dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.470490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0c793-c378-4cf1-99c1-f2e72e9468dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.475688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0c793-c378-4cf1-99c1-f2e72e9468dd admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.481080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c42c6f-4e5b-48a9-ac8e-1a089b0dbbe7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.484744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1ba56-74f5-45d5-be83-eacf610322fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.485343 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1ba56-74f5-45d5-be83-eacf610322fc service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.485663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1ba56-74f5-45d5-be83-eacf610322fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.486082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 793/1595] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.486200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1ba56-74f5-45d5-be83-eacf610322fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.487285 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0c793-c378-4cf1-99c1-f2e72e9468dd admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.498875 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1ba56-74f5-45d5-be83-eacf610322fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.500368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1ba56-74f5-45d5-be83-eacf610322fc service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.501626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0c793-c378-4cf1-99c1-f2e72e9468dd admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.508783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eead51-6456-44a2-a580-d0d8329cfc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.509400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eead51-6456-44a2-a580-d0d8329cfc93 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.509718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eead51-6456-44a2-a580-d0d8329cfc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.509993 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eead51-6456-44a2-a580-d0d8329cfc93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.510647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eead51-6456-44a2-a580-d0d8329cfc93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.512246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eead51-6456-44a2-a580-d0d8329cfc93 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.514865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 803/1596] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:38.515573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1ba56-74f5-45d5-be83-eacf610322fc service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.526364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1ba56-74f5-45d5-be83-eacf610322fc service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.530791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 794/1597] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.530791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eead51-6456-44a2-a580-d0d8329cfc93 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.534685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d64fd-ad75-43e8-aa0f-63b8553ce14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.535367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d64fd-ad75-43e8-aa0f-63b8553ce14f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.535651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d64fd-ad75-43e8-aa0f-63b8553ce14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.535927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d64fd-ad75-43e8-aa0f-63b8553ce14f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.536841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d64fd-ad75-43e8-aa0f-63b8553ce14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.538397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d64fd-ad75-43e8-aa0f-63b8553ce14f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.544382 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eead51-6456-44a2-a580-d0d8329cfc93 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.550202 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5518f3c-012b-452f-b41c-7d11813ebbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.556172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 804/1598] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:38.556317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5518f3c-012b-452f-b41c-7d11813ebbaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.556317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5518f3c-012b-452f-b41c-7d11813ebbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.556511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d64fd-ad75-43e8-aa0f-63b8553ce14f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.556612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5518f3c-012b-452f-b41c-7d11813ebbaf admin admin] PATH_INFO: `/v3/projects/9e4e6413aa9c4928ba69c6efccc64f7a/users/7a1288d5b9334a2e8b2c9bd1a2793376/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.563060 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d64fd-ad75-43e8-aa0f-63b8553ce14f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.568800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206123f-cecd-48e8-8b58-7b313d806ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.569462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206123f-cecd-48e8-8b58-7b313d806ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.570529 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206123f-cecd-48e8-8b58-7b313d806ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.570720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206123f-cecd-48e8-8b58-7b313d806ddf admin admin] PATH_INFO: `/v3/projects/bfec9cc41c7346acb8b4405c8603b07c/users/0f93cab77f2b4bfd839c4e08ddfa1da7/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.575096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5518f3c-012b-452f-b41c-7d11813ebbaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e4e6413aa9c4928ba69c6efccc64f7a, user_id=7a1288d5b9334a2e8b2c9bd1a2793376, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.576331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-b5518f3c-012b-452f-b41c-7d11813ebbaf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:48:38.576331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Aug 09 18:48:38.576331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: Aug 09 18:48:38.582488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5518f3c-012b-452f-b41c-7d11813ebbaf admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.587745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206123f-cecd-48e8-8b58-7b313d806ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfec9cc41c7346acb8b4405c8603b07c, user_id=0f93cab77f2b4bfd839c4e08ddfa1da7, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.591060 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206123f-cecd-48e8-8b58-7b313d806ddf admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.699367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 795/1599] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:38 2022] PUT /identity/v3/projects/9e4e6413aa9c4928ba69c6efccc64f7a/users/7a1288d5b9334a2e8b2c9bd1a2793376/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:38.701895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5518f3c-012b-452f-b41c-7d11813ebbaf admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.732918 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 805/1600] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:38 2022] PUT /identity/v3/projects/bfec9cc41c7346acb8b4405c8603b07c/users/0f93cab77f2b4bfd839c4e08ddfa1da7/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:38.736318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206123f-cecd-48e8-8b58-7b313d806ddf admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.768766 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5518f3c-012b-452f-b41c-7d11813ebbaf admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.804400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206123f-cecd-48e8-8b58-7b313d806ddf admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:38.811978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f1e33-1fc9-4200-ac47-6af990d5ccee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.812714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f1e33-1fc9-4200-ac47-6af990d5ccee admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.813001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f1e33-1fc9-4200-ac47-6af990d5ccee admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.813331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f1e33-1fc9-4200-ac47-6af990d5ccee admin admin] PATH_INFO: `/v3/projects/d3c7915d78334e95a41a32f0f3b5b084/users/04da0f4276d6457c889ecb3006842e9c/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.832259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f1e33-1fc9-4200-ac47-6af990d5ccee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c7915d78334e95a41a32f0f3b5b084, user_id=04da0f4276d6457c889ecb3006842e9c, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.834092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92071aec-dd83-4ebd-91e2-bf3aa191791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:38.834092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92071aec-dd83-4ebd-91e2-bf3aa191791c admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:38.834092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92071aec-dd83-4ebd-91e2-bf3aa191791c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:38.834092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92071aec-dd83-4ebd-91e2-bf3aa191791c admin admin] PATH_INFO: `/v3/projects/4d8f60ca836d43399989fb32ad78d27d/users/a6e48ee30c61424ba0c82ac4fc5cd06c/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:38.835223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f1e33-1fc9-4200-ac47-6af990d5ccee admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.854829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92071aec-dd83-4ebd-91e2-bf3aa191791c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d8f60ca836d43399989fb32ad78d27d, user_id=a6e48ee30c61424ba0c82ac4fc5cd06c, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:38.857778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92071aec-dd83-4ebd-91e2-bf3aa191791c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:38.939098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 806/1601] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:38 2022] PUT /identity/v3/projects/d3c7915d78334e95a41a32f0f3b5b084/users/04da0f4276d6457c889ecb3006842e9c/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:38.939995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f1e33-1fc9-4200-ac47-6af990d5ccee admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:38.982809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:39.001584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f1e33-1fc9-4200-ac47-6af990d5ccee admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:39.008442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 796/1602] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:38 2022] PUT /identity/v3/projects/4d8f60ca836d43399989fb32ad78d27d/users/a6e48ee30c61424ba0c82ac4fc5cd06c/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:39.009447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92071aec-dd83-4ebd-91e2-bf3aa191791c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:39.043261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:39.061912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6297bbd-1456-4875-9019-80264b0a64c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:39.062578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6297bbd-1456-4875-9019-80264b0a64c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.062859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6297bbd-1456-4875-9019-80264b0a64c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.063180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6297bbd-1456-4875-9019-80264b0a64c1 admin admin] PATH_INFO: `/v3/projects/ad76f9f6c7b44b8c88c02b65f4ac3d87/users/cdc9777c9c4c461b849bec17dc82997d/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.072539 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92071aec-dd83-4ebd-91e2-bf3aa191791c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:39.077364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd2663-ad78-497a-896f-c07c39b4455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:39.080663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6297bbd-1456-4875-9019-80264b0a64c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad76f9f6c7b44b8c88c02b65f4ac3d87, user_id=cdc9777c9c4c461b849bec17dc82997d, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:39.080824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd2663-ad78-497a-896f-c07c39b4455b admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.080824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd2663-ad78-497a-896f-c07c39b4455b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.080824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd2663-ad78-497a-896f-c07c39b4455b admin admin] PATH_INFO: `/v3/projects/6d738e0a61184b4db58e2607d72cca0a/users/ef8af9fd9e844ca4be10caa880357a4f/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.081124 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6297bbd-1456-4875-9019-80264b0a64c1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:39.093766 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd2663-ad78-497a-896f-c07c39b4455b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d738e0a61184b4db58e2607d72cca0a, user_id=ef8af9fd9e844ca4be10caa880357a4f, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:39.096398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd2663-ad78-497a-896f-c07c39b4455b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:39.211433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 807/1603] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:38 2022] PUT /identity/v3/projects/ad76f9f6c7b44b8c88c02b65f4ac3d87/users/cdc9777c9c4c461b849bec17dc82997d/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:39.211433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6297bbd-1456-4875-9019-80264b0a64c1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:39.274591 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6297bbd-1456-4875-9019-80264b0a64c1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:39.279178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 797/1604] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:39 2022] PUT /identity/v3/projects/6d738e0a61184b4db58e2607d72cca0a/users/ef8af9fd9e844ca4be10caa880357a4f/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:39.284476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcfe36-c7b6-4ea0-8745-7ed30246967e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.284617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcfe36-c7b6-4ea0-8745-7ed30246967e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.284887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcfe36-c7b6-4ea0-8745-7ed30246967e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.312805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a9fcfe36-c7b6-4ea0-8745-7ed30246967e None None] MFA Rules not processed for user `7a1288d5b9334a2e8b2c9bd1a2793376`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:39.314154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a9fcfe36-c7b6-4ea0-8745-7ed30246967e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:39.333401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05efa62-0f8f-47a7-8eee-e0e376923437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:39.334104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05efa62-0f8f-47a7-8eee-e0e376923437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.334389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05efa62-0f8f-47a7-8eee-e0e376923437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.334713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05efa62-0f8f-47a7-8eee-e0e376923437 admin admin] PATH_INFO: `/v3/projects/0cd0e883aec7498ca54ea52fa547edde/users/bfc90a2b0c054c93afdf725381859dd9/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.349468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05efa62-0f8f-47a7-8eee-e0e376923437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd0e883aec7498ca54ea52fa547edde, user_id=bfc90a2b0c054c93afdf725381859dd9, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:39.352259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05efa62-0f8f-47a7-8eee-e0e376923437 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:39.381464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 798/1605] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:39.382847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2f973-3b29-48e4-8209-74a059f395e6 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.383151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2f973-3b29-48e4-8209-74a059f395e6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.383490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2f973-3b29-48e4-8209-74a059f395e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.412259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6ab2f973-3b29-48e4-8209-74a059f395e6 None None] MFA Rules not processed for user `0f93cab77f2b4bfd839c4e08ddfa1da7`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:39.413606 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6ab2f973-3b29-48e4-8209-74a059f395e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:39.432387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:39.438381 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 808/1606] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:39 2022] PUT /identity/v3/projects/0cd0e883aec7498ca54ea52fa547edde/users/bfc90a2b0c054c93afdf725381859dd9/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:39.439545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f3e67-6f0e-47ec-ad86-90f06b2fc664 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.440094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f3e67-6f0e-47ec-ad86-90f06b2fc664 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.440378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f3e67-6f0e-47ec-ad86-90f06b2fc664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.465081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-111f3e67-6f0e-47ec-ad86-90f06b2fc664 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:39.466575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-111f3e67-6f0e-47ec-ad86-90f06b2fc664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:39.493986 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 799/1607] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:39 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:39.497689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23197baa-cec3-41e9-9fcb-0a9af7673294 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.497689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23197baa-cec3-41e9-9fcb-0a9af7673294 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.497689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23197baa-cec3-41e9-9fcb-0a9af7673294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.517087 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-23197baa-cec3-41e9-9fcb-0a9af7673294 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:39.518441 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-23197baa-cec3-41e9-9fcb-0a9af7673294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:39.536751 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 809/1608] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:39.538356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19f531-4484-4dbf-9776-8ed760ed9105 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.538718 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19f531-4484-4dbf-9776-8ed760ed9105 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.539062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19f531-4484-4dbf-9776-8ed760ed9105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.555774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:39.569244 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4f19f531-4484-4dbf-9776-8ed760ed9105 None None] MFA Rules not processed for user `cdc9777c9c4c461b849bec17dc82997d`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:39.570567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4f19f531-4484-4dbf-9776-8ed760ed9105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:39.586168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 800/1609] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:39.587422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570046c-ed4a-4d22-bf12-115d6dff0047 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.587711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570046c-ed4a-4d22-bf12-115d6dff0047 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.588088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570046c-ed4a-4d22-bf12-115d6dff0047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.607286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5570046c-ed4a-4d22-bf12-115d6dff0047 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:39.608611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5570046c-ed4a-4d22-bf12-115d6dff0047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:39.617293 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 801/1610] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:39.618097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570046c-ed4a-4d22-bf12-115d6dff0047 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:39.627629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570046c-ed4a-4d22-bf12-115d6dff0047 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:39.633596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d278ea0-1717-4ca3-aa9c-869fc61c14ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:39.634228 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d278ea0-1717-4ca3-aa9c-869fc61c14ce admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.634483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d278ea0-1717-4ca3-aa9c-869fc61c14ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.634756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d278ea0-1717-4ca3-aa9c-869fc61c14ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.635382 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d278ea0-1717-4ca3-aa9c-869fc61c14ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:39.637301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d278ea0-1717-4ca3-aa9c-869fc61c14ce admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:39.643558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 810/1611] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:39.644867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b8b9d-ca11-487a-bb8f-ddd24cb79b7a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.645154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b8b9d-ca11-487a-bb8f-ddd24cb79b7a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.645568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b8b9d-ca11-487a-bb8f-ddd24cb79b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.673723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-043b8b9d-ca11-487a-bb8f-ddd24cb79b7a None None] MFA Rules not processed for user `bfc90a2b0c054c93afdf725381859dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:39.675041 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-043b8b9d-ca11-487a-bb8f-ddd24cb79b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:39.753362 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 811/1612] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:39.753362 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b8b9d-ca11-487a-bb8f-ddd24cb79b7a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:39.758882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 802/1613] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:39 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:39.760047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d278ea0-1717-4ca3-aa9c-869fc61c14ce admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:39.763583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b8b9d-ca11-487a-bb8f-ddd24cb79b7a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:39.780751 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:39.819172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d278ea0-1717-4ca3-aa9c-869fc61c14ce admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:39.821096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dd006-a118-423a-a3a0-57eb37e0bd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:39.821736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dd006-a118-423a-a3a0-57eb37e0bd62 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.822016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dd006-a118-423a-a3a0-57eb37e0bd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.822284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dd006-a118-423a-a3a0-57eb37e0bd62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.823038 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dd006-a118-423a-a3a0-57eb37e0bd62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:39.823161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55be3d-637e-4a7f-be3f-3fca7b4a8fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:39.823673 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55be3d-637e-4a7f-be3f-3fca7b4a8fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.823958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55be3d-637e-4a7f-be3f-3fca7b4a8fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.824235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55be3d-637e-4a7f-be3f-3fca7b4a8fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.824964 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dd006-a118-423a-a3a0-57eb37e0bd62 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:39.833942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55be3d-637e-4a7f-be3f-3fca7b4a8fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:39.835382 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55be3d-637e-4a7f-be3f-3fca7b4a8fc8 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:39.845041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 803/1614] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:39.845995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55be3d-637e-4a7f-be3f-3fca7b4a8fc8 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:39.854036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55be3d-637e-4a7f-be3f-3fca7b4a8fc8 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:39.857966 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129401d-37bc-42e3-bb04-2de824a67d2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:39.858554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129401d-37bc-42e3-bb04-2de824a67d2c service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.858832 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129401d-37bc-42e3-bb04-2de824a67d2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.859101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129401d-37bc-42e3-bb04-2de824a67d2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.868980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129401d-37bc-42e3-bb04-2de824a67d2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:39.870446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129401d-37bc-42e3-bb04-2de824a67d2c service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:39.881224 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 804/1615] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:39.882053 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129401d-37bc-42e3-bb04-2de824a67d2c service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:39.889982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129401d-37bc-42e3-bb04-2de824a67d2c service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:39.893797 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065d207-4b20-4df2-b8e1-7707f123bf21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:39.894408 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065d207-4b20-4df2-b8e1-7707f123bf21 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.894688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065d207-4b20-4df2-b8e1-7707f123bf21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.894959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065d207-4b20-4df2-b8e1-7707f123bf21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.904770 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065d207-4b20-4df2-b8e1-7707f123bf21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:39.906214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065d207-4b20-4df2-b8e1-7707f123bf21 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:39.916450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 805/1616] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:39.917207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065d207-4b20-4df2-b8e1-7707f123bf21 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:39.926952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065d207-4b20-4df2-b8e1-7707f123bf21 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:39.931889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b977081-289f-45fa-b710-3254f9434c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:39.932479 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b977081-289f-45fa-b710-3254f9434c48 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:39.932761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b977081-289f-45fa-b710-3254f9434c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:39.933032 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b977081-289f-45fa-b710-3254f9434c48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:39.933678 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b977081-289f-45fa-b710-3254f9434c48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:39.935554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b977081-289f-45fa-b710-3254f9434c48 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:39.968897 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 812/1617] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:39 2022] POST /identity/v3/projects => generated 357 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:39.969935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dd006-a118-423a-a3a0-57eb37e0bd62 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.043780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dd006-a118-423a-a3a0-57eb37e0bd62 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:40.050177 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba57022-8433-4fa1-a9b7-457913ac67ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:40.051397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba57022-8433-4fa1-a9b7-457913ac67ca admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:40.052392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba57022-8433-4fa1-a9b7-457913ac67ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:40.052392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba57022-8433-4fa1-a9b7-457913ac67ca admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:40.052769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba57022-8433-4fa1-a9b7-457913ac67ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:40.054794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba57022-8433-4fa1-a9b7-457913ac67ca admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:40.106198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 806/1618] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:39 2022] POST /identity/v3/projects => generated 365 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:40.106198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b977081-289f-45fa-b710-3254f9434c48 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.183230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b977081-289f-45fa-b710-3254f9434c48 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:40.188617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde46526-4cf6-4733-975d-d104319a2b20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:40.197965 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde46526-4cf6-4733-975d-d104319a2b20 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:40.198317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde46526-4cf6-4733-975d-d104319a2b20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:40.198602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde46526-4cf6-4733-975d-d104319a2b20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:40.199617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 813/1619] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:39 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:40.201189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba57022-8433-4fa1-a9b7-457913ac67ca admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.215909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:40.218239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba57022-8433-4fa1-a9b7-457913ac67ca admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:40.287581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde46526-4cf6-4733-975d-d104319a2b20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:40.289905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde46526-4cf6-4733-975d-d104319a2b20 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:40.301863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 807/1620] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:40.302325 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c79e5-dadd-400f-bb7b-13c70fb2ceba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:40.303055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c79e5-dadd-400f-bb7b-13c70fb2ceba admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:40.303422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde46526-4cf6-4733-975d-d104319a2b20 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.303567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c79e5-dadd-400f-bb7b-13c70fb2ceba admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:40.304010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c79e5-dadd-400f-bb7b-13c70fb2ceba admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:40.304767 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c79e5-dadd-400f-bb7b-13c70fb2ceba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:40.307009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c79e5-dadd-400f-bb7b-13c70fb2ceba admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:40.315664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde46526-4cf6-4733-975d-d104319a2b20 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:40.323481 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6e094-9b58-42cf-b09a-0625293455f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:40.323987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6e094-9b58-42cf-b09a-0625293455f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:40.324362 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6e094-9b58-42cf-b09a-0625293455f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:40.324790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6e094-9b58-42cf-b09a-0625293455f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:40.325566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6e094-9b58-42cf-b09a-0625293455f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:40.340404 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6e094-9b58-42cf-b09a-0625293455f8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:40.419749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 814/1621] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:40 2022] POST /identity/v3/users => generated 285 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:40.420611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c79e5-dadd-400f-bb7b-13c70fb2ceba admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.440068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c79e5-dadd-400f-bb7b-13c70fb2ceba admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:40.453673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939dade-e2e0-4e34-b6ac-87fa31c6d1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:40.455934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939dade-e2e0-4e34-b6ac-87fa31c6d1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:40.455934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939dade-e2e0-4e34-b6ac-87fa31c6d1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:40.455934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939dade-e2e0-4e34-b6ac-87fa31c6d1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:40.455934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939dade-e2e0-4e34-b6ac-87fa31c6d1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:40.457761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939dade-e2e0-4e34-b6ac-87fa31c6d1ba admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:40.483462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 815/1622] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:40.483462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939dade-e2e0-4e34-b6ac-87fa31c6d1ba admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.496000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939dade-e2e0-4e34-b6ac-87fa31c6d1ba admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:40.504508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fc4b2-9679-4ee7-9d54-510b51f3a6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:40.504999 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fc4b2-9679-4ee7-9d54-510b51f3a6bd admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:40.505350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fc4b2-9679-4ee7-9d54-510b51f3a6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:40.505731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fc4b2-9679-4ee7-9d54-510b51f3a6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:40.506627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769fc4b2-9679-4ee7-9d54-510b51f3a6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:40.508174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769fc4b2-9679-4ee7-9d54-510b51f3a6bd admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:40.521153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 816/1623] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:40.521153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fc4b2-9679-4ee7-9d54-510b51f3a6bd admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.526808 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:40.530772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 808/1624] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:40 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:40.531762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6e094-9b58-42cf-b09a-0625293455f8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.531901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fc4b2-9679-4ee7-9d54-510b51f3a6bd admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:40.538803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2881b-a8ce-4a3e-8999-31b5dab1de89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:40.539414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2881b-a8ce-4a3e-8999-31b5dab1de89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:40.539699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2881b-a8ce-4a3e-8999-31b5dab1de89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:40.539977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2881b-a8ce-4a3e-8999-31b5dab1de89 admin admin] PATH_INFO: `/v3/projects/786a9cab00e24528adeb75420779007f/users/8c3ddbedbda24291a35d9c24dbefa3a6/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:40.543657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6e094-9b58-42cf-b09a-0625293455f8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:40.549353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9d0bf-65db-4f4c-9e0d-eb70278105ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:40.549353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9d0bf-65db-4f4c-9e0d-eb70278105ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:40.553571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9d0bf-65db-4f4c-9e0d-eb70278105ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:40.553571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9d0bf-65db-4f4c-9e0d-eb70278105ca admin admin] PATH_INFO: `/v3/projects/fead135916ef452da4ab72d4ca01fff9/users/a1e9d7912646464cb9a860dd692c829a/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:40.558052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2881b-a8ce-4a3e-8999-31b5dab1de89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=786a9cab00e24528adeb75420779007f, user_id=8c3ddbedbda24291a35d9c24dbefa3a6, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:40.562027 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2881b-a8ce-4a3e-8999-31b5dab1de89 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:40.574183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9d0bf-65db-4f4c-9e0d-eb70278105ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=fead135916ef452da4ab72d4ca01fff9, user_id=a1e9d7912646464cb9a860dd692c829a, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:40.576972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9d0bf-65db-4f4c-9e0d-eb70278105ca admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:40.688051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 817/1625] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:40 2022] PUT /identity/v3/projects/786a9cab00e24528adeb75420779007f/users/8c3ddbedbda24291a35d9c24dbefa3a6/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:40.688968 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2881b-a8ce-4a3e-8999-31b5dab1de89 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.698169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 809/1626] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:40 2022] PUT /identity/v3/projects/fead135916ef452da4ab72d4ca01fff9/users/a1e9d7912646464cb9a860dd692c829a/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:40.701122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9d0bf-65db-4f4c-9e0d-eb70278105ca admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.819110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2881b-a8ce-4a3e-8999-31b5dab1de89 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:40.896710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9d0bf-65db-4f4c-9e0d-eb70278105ca admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:40.901895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cf4ef-9316-49c9-807f-404995cc754f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:40.902606 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cf4ef-9316-49c9-807f-404995cc754f admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:40.903039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cf4ef-9316-49c9-807f-404995cc754f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:40.903286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cf4ef-9316-49c9-807f-404995cc754f admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:40.904251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cf4ef-9316-49c9-807f-404995cc754f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:40.905736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cf4ef-9316-49c9-807f-404995cc754f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:40.917853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 810/1627] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:40.918727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cf4ef-9316-49c9-807f-404995cc754f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.927708 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41f652-0c90-46e4-add1-10b7b45719c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:40.928455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41f652-0c90-46e4-add1-10b7b45719c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:40.928860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41f652-0c90-46e4-add1-10b7b45719c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:40.929230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41f652-0c90-46e4-add1-10b7b45719c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:40.930085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41f652-0c90-46e4-add1-10b7b45719c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:40.933157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cf4ef-9316-49c9-807f-404995cc754f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:40.938111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e555b2-a749-4e87-8024-01fdae3f4b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:40.942564 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e555b2-a749-4e87-8024-01fdae3f4b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:40.942564 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e555b2-a749-4e87-8024-01fdae3f4b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:40.942564 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e555b2-a749-4e87-8024-01fdae3f4b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:40.942564 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e555b2-a749-4e87-8024-01fdae3f4b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:40.943268 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e555b2-a749-4e87-8024-01fdae3f4b26 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:40.954802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 811/1628] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:40.955782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e555b2-a749-4e87-8024-01fdae3f4b26 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.958040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41f652-0c90-46e4-add1-10b7b45719c5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:40.987577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e555b2-a749-4e87-8024-01fdae3f4b26 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:40.988291 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 818/1629] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:40.988291 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41f652-0c90-46e4-add1-10b7b45719c5 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:40.992932 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a40e27-bcf2-4acf-b07a-20e2742e3c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:40.993643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a40e27-bcf2-4acf-b07a-20e2742e3c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:40.993931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a40e27-bcf2-4acf-b07a-20e2742e3c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:40.994363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a40e27-bcf2-4acf-b07a-20e2742e3c17 admin admin] PATH_INFO: `/v3/projects/786a9cab00e24528adeb75420779007f/users/8c3ddbedbda24291a35d9c24dbefa3a6/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:41.000278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a40e27-bcf2-4acf-b07a-20e2742e3c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=786a9cab00e24528adeb75420779007f, user_id=8c3ddbedbda24291a35d9c24dbefa3a6, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:41.008373 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a40e27-bcf2-4acf-b07a-20e2742e3c17 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:41.009441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41f652-0c90-46e4-add1-10b7b45719c5 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:41.023442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba0c36-7306-4efc-82cf-8c0a88ab8b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:41.024140 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba0c36-7306-4efc-82cf-8c0a88ab8b45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:41.024537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba0c36-7306-4efc-82cf-8c0a88ab8b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:41.024815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba0c36-7306-4efc-82cf-8c0a88ab8b45 admin admin] PATH_INFO: `/v3/projects/fead135916ef452da4ab72d4ca01fff9/users/a1e9d7912646464cb9a860dd692c829a/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:41.031457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba0c36-7306-4efc-82cf-8c0a88ab8b45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fead135916ef452da4ab72d4ca01fff9, user_id=a1e9d7912646464cb9a860dd692c829a, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:41.044710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 812/1630] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:40 2022] PUT /identity/v3/projects/786a9cab00e24528adeb75420779007f/users/8c3ddbedbda24291a35d9c24dbefa3a6/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:41.045166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba0c36-7306-4efc-82cf-8c0a88ab8b45 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:41.045251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a40e27-bcf2-4acf-b07a-20e2742e3c17 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:41.064566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 819/1631] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:40 2022] PUT /identity/v3/projects/fead135916ef452da4ab72d4ca01fff9/users/a1e9d7912646464cb9a860dd692c829a/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:41.066666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77b6ae-3347-4de3-8eb3-5db45e444067 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:41.067694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77b6ae-3347-4de3-8eb3-5db45e444067 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:41.067987 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77b6ae-3347-4de3-8eb3-5db45e444067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:41.153490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1b77b6ae-3347-4de3-8eb3-5db45e444067 None None] MFA Rules not processed for user `8c3ddbedbda24291a35d9c24dbefa3a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:41.155303 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1b77b6ae-3347-4de3-8eb3-5db45e444067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:41.195018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:41.229448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a40e27-bcf2-4acf-b07a-20e2742e3c17 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:41.304714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:41.329413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 820/1632] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:41.420453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648b0fd-8b17-4ea7-a754-ed8a11e5fa2e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:41.420453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648b0fd-8b17-4ea7-a754-ed8a11e5fa2e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:41.420453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648b0fd-8b17-4ea7-a754-ed8a11e5fa2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:41.420453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-9648b0fd-8b17-4ea7-a754-ed8a11e5fa2e None None] MFA Rules not processed for user `a1e9d7912646464cb9a860dd692c829a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:41.420453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9648b0fd-8b17-4ea7-a754-ed8a11e5fa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:41.433045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc3090-9ef3-455a-bcf4-1921d590c0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:41.434175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc3090-9ef3-455a-bcf4-1921d590c0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:41.434598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc3090-9ef3-455a-bcf4-1921d590c0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:41.435096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc3090-9ef3-455a-bcf4-1921d590c0a9 admin admin] PATH_INFO: `/v3/projects/c5fa2cda24ed4953876fe8af1fa69a4f/users/edc94bc025714599941e66096fc10acf/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:41.461701 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc3090-9ef3-455a-bcf4-1921d590c0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5fa2cda24ed4953876fe8af1fa69a4f, user_id=edc94bc025714599941e66096fc10acf, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:41.464895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc3090-9ef3-455a-bcf4-1921d590c0a9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:41.548063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 813/1633] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:41 2022] PUT /identity/v3/projects/c5fa2cda24ed4953876fe8af1fa69a4f/users/edc94bc025714599941e66096fc10acf/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:41.549174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc3090-9ef3-455a-bcf4-1921d590c0a9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:41.573052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 821/1634] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:41 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:41.574787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648b0fd-8b17-4ea7-a754-ed8a11e5fa2e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:41.597652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648b0fd-8b17-4ea7-a754-ed8a11e5fa2e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:41.743980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc3090-9ef3-455a-bcf4-1921d590c0a9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:41.747000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486195f4-e83a-407e-b318-d7b3478f9161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:41.747810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486195f4-e83a-407e-b318-d7b3478f9161 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:41.748169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486195f4-e83a-407e-b318-d7b3478f9161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:41.748887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c65ac7-1deb-4747-a766-497e020de65c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:41.749457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486195f4-e83a-407e-b318-d7b3478f9161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:41.749753 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c65ac7-1deb-4747-a766-497e020de65c service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:41.750101 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486195f4-e83a-407e-b318-d7b3478f9161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:41.751090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c65ac7-1deb-4747-a766-497e020de65c service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:41.751473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c65ac7-1deb-4747-a766-497e020de65c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:41.755346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486195f4-e83a-407e-b318-d7b3478f9161 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:41.786880 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 822/1635] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:41.788008 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486195f4-e83a-407e-b318-d7b3478f9161 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:41.815782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486195f4-e83a-407e-b318-d7b3478f9161 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:41.827465 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2d767-7527-4c35-b6e1-57dfe50b2b39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:41.828240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2d767-7527-4c35-b6e1-57dfe50b2b39 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:41.835546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2d767-7527-4c35-b6e1-57dfe50b2b39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:41.835546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2d767-7527-4c35-b6e1-57dfe50b2b39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:41.849430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c65ac7-1deb-4747-a766-497e020de65c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:41.851217 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c65ac7-1deb-4747-a766-497e020de65c service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:41.866136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 814/1636] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:41.867224 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c65ac7-1deb-4747-a766-497e020de65c service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:41.882883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c65ac7-1deb-4747-a766-497e020de65c service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:41.890784 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d886b-b813-4e2b-a21a-ce3291a8a6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:41.904423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d886b-b813-4e2b-a21a-ce3291a8a6f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:41.904423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d886b-b813-4e2b-a21a-ce3291a8a6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:41.904423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d886b-b813-4e2b-a21a-ce3291a8a6f3 admin admin] PATH_INFO: `/v3/projects/c5fa2cda24ed4953876fe8af1fa69a4f/users/edc94bc025714599941e66096fc10acf/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:41.904423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d886b-b813-4e2b-a21a-ce3291a8a6f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5fa2cda24ed4953876fe8af1fa69a4f, user_id=edc94bc025714599941e66096fc10acf, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:41.904423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d886b-b813-4e2b-a21a-ce3291a8a6f3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:41.912979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2d767-7527-4c35-b6e1-57dfe50b2b39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:41.914740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2d767-7527-4c35-b6e1-57dfe50b2b39 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:41.926704 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 815/1637] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:41 2022] PUT /identity/v3/projects/c5fa2cda24ed4953876fe8af1fa69a4f/users/edc94bc025714599941e66096fc10acf/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:41.929263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 823/1638] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:41 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:41.939287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747c653-cd79-4747-8272-cf5cb0172abb None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:41.940913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747c653-cd79-4747-8272-cf5cb0172abb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:41.941307 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747c653-cd79-4747-8272-cf5cb0172abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:41.982732 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9747c653-cd79-4747-8272-cf5cb0172abb None None] MFA Rules not processed for user `edc94bc025714599941e66096fc10acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:41.984625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9747c653-cd79-4747-8272-cf5cb0172abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:42.074712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:42.113315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 816/1639] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:48:42.228714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2d767-7527-4c35-b6e1-57dfe50b2b39 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:42.238969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2d767-7527-4c35-b6e1-57dfe50b2b39 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:42.278792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:42.313363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e644fa9-39e1-4897-a07e-44946580bb7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:42.314403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e644fa9-39e1-4897-a07e-44946580bb7a service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:42.314884 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e644fa9-39e1-4897-a07e-44946580bb7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:42.315318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e644fa9-39e1-4897-a07e-44946580bb7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:42.338992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e644fa9-39e1-4897-a07e-44946580bb7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:42.340795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e644fa9-39e1-4897-a07e-44946580bb7a service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:42.355943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 824/1640] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:42.502680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747c653-cd79-4747-8272-cf5cb0172abb None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:42.514048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747c653-cd79-4747-8272-cf5cb0172abb None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:42.520535 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4753e-7e71-482f-8df9-58b2fdeb7ae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:42.520535 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4753e-7e71-482f-8df9-58b2fdeb7ae6 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:42.520535 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4753e-7e71-482f-8df9-58b2fdeb7ae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:42.520535 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4753e-7e71-482f-8df9-58b2fdeb7ae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:42.656822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4753e-7e71-482f-8df9-58b2fdeb7ae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:42.658500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4753e-7e71-482f-8df9-58b2fdeb7ae6 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:43.045982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 817/1641] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:43.047879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e644fa9-39e1-4897-a07e-44946580bb7a service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:43.047879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e644fa9-39e1-4897-a07e-44946580bb7a service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:43.047879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a1caf-d721-436f-aeeb-1df4fa3d3c98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:43.047879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a1caf-d721-436f-aeeb-1df4fa3d3c98 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:43.048744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a1caf-d721-436f-aeeb-1df4fa3d3c98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:43.048744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a1caf-d721-436f-aeeb-1df4fa3d3c98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:43.124088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a1caf-d721-436f-aeeb-1df4fa3d3c98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:43.125957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a1caf-d721-436f-aeeb-1df4fa3d3c98 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:43.142057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 825/1642] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:44.311012 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4753e-7e71-482f-8df9-58b2fdeb7ae6 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:44.437295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4ead-0084-4e45-b57e-8d41f36ec470 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:44.437898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4ead-0084-4e45-b57e-8d41f36ec470 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:44.438256 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4ead-0084-4e45-b57e-8d41f36ec470 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:44.440161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 826/1643] 10.222.0.57 () {54 vars in 962 bytes} [Tue Aug 9 18:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:48:44.448731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4753e-7e71-482f-8df9-58b2fdeb7ae6 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:44.452162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa2e30-22b6-4255-ab50-3dd92a475f1d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:44.452450 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa2e30-22b6-4255-ab50-3dd92a475f1d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:44.452728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa2e30-22b6-4255-ab50-3dd92a475f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:44.469045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:44.469747 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:44.470078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:44.470304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] PATH_INFO: `/v3/users/7a1288d5b9334a2e8b2c9bd1a2793376` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:44.471443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a1288d5b9334a2e8b2c9bd1a2793376)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:44.472502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: WARNING py.warnings [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:48:44.472502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: warnings.warn(msg) Aug 09 18:48:44.472502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: Aug 09 18:48:44.473825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:44.540230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-3aaa2e30-22b6-4255-ab50-3dd92a475f1d None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:44.541918 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3aaa2e30-22b6-4255-ab50-3dd92a475f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:44.640003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 827/1644] 10.222.0.57 () {58 vars in 1059 bytes} [Tue Aug 9 18:48:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:44.644856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010b104-fa20-47ff-803f-5eabf4debf39 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:44.645575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010b104-fa20-47ff-803f-5eabf4debf39 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:44.650111 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010b104-fa20-47ff-803f-5eabf4debf39 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:44.654989 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 828/1645] 10.222.0.57 () {54 vars in 912 bytes} [Tue Aug 9 18:48:44 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:48:44.656464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606b83c-e0f3-453a-960f-30c66994f85e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:44.656796 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606b83c-e0f3-453a-960f-30c66994f85e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:44.657080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606b83c-e0f3-453a-960f-30c66994f85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:44.680794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4606b83c-e0f3-453a-960f-30c66994f85e None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:44.682046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4606b83c-e0f3-453a-960f-30c66994f85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:44.707290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 829/1646] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:48:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:44.722033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606b83c-e0f3-453a-960f-30c66994f85e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:44.722257 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:44.751395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606b83c-e0f3-453a-960f-30c66994f85e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:44.755932 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a1288d5b9334a2e8b2c9bd1a2793376'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:44.760774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d790a3-4b3a-42b7-b04f-fce34b08a449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:44.761411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d790a3-4b3a-42b7-b04f-fce34b08a449 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:44.761716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d790a3-4b3a-42b7-b04f-fce34b08a449 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:44.762499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d790a3-4b3a-42b7-b04f-fce34b08a449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:44.780155 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d790a3-4b3a-42b7-b04f-fce34b08a449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:44.782144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d790a3-4b3a-42b7-b04f-fce34b08a449 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:44.798541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 830/1647] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:44.819337 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a1288d5b9334a2e8b2c9bd1a2793376'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:44.820430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a1288d5b9334a2e8b2c9bd1a2793376'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:44.843859 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a1288d5b9334a2e8b2c9bd1a2793376'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:44.923141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a1288d5b9334a2e8b2c9bd1a2793376'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:44.945227 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 818/1648] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:48:44 2022] DELETE /identity/v3/users/7a1288d5b9334a2e8b2c9bd1a2793376 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:44.949494 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8ba5f-8fec-4c9d-895f-39de6b8d0d29 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:44.949820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8ba5f-8fec-4c9d-895f-39de6b8d0d29 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:44.950097 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8ba5f-8fec-4c9d-895f-39de6b8d0d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:45.000801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0fc8ba5f-8fec-4c9d-895f-39de6b8d0d29 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:45.002161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0fc8ba5f-8fec-4c9d-895f-39de6b8d0d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:45.007643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:45.188385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 831/1649] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:45.266646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:45.268297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:45.399425 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e19485-bda2-4512-848c-e0c8021d6814 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:45.408554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c847bb-6b60-473f-991b-19208337235d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:45.408773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c847bb-6b60-473f-991b-19208337235d service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:45.409038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c847bb-6b60-473f-991b-19208337235d service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:45.409315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c847bb-6b60-473f-991b-19208337235d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:45.424187 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c847bb-6b60-473f-991b-19208337235d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:45.426678 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c847bb-6b60-473f-991b-19208337235d service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:45.462115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 819/1650] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:47.022812 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8ba5f-8fec-4c9d-895f-39de6b8d0d29 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:47.024442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0fc8ba5f-8fec-4c9d-895f-39de6b8d0d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:47.044634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8ba5f-8fec-4c9d-895f-39de6b8d0d29 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:47.054248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:47.055421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:47.056153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:47.056818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] PATH_INFO: `/v3/projects/9e4e6413aa9c4928ba69c6efccc64f7a` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:47.058712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e4e6413aa9c4928ba69c6efccc64f7a)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:47.062212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: WARNING py.warnings [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 18:48:47.062212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: warnings.warn(msg) Aug 09 18:48:47.062212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: Aug 09 18:48:47.063467 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:47.320158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e4e6413aa9c4928ba69c6efccc64f7a', 'request_id': 'req-fb10b140-e4fa-4a4c-813a-4d7e915902cd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:47.336089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e4e6413aa9c4928ba69c6efccc64f7a', 'request_id': 'req-fb10b140-e4fa-4a4c-813a-4d7e915902cd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:47.389285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e4e6413aa9c4928ba69c6efccc64f7a', 'request_id': 'req-fb10b140-e4fa-4a4c-813a-4d7e915902cd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:47.503871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] The token cache is being invalidate because project 9e4e6413aa9c4928ba69c6efccc64f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:48:47.504220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:47.505137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-fb10b140-e4fa-4a4c-813a-4d7e915902cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:47.510100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 832/1651] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:48:47 2022] DELETE /identity/v3/projects/9e4e6413aa9c4928ba69c6efccc64f7a => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:47.514365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c847bb-6b60-473f-991b-19208337235d service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:47.532211 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c847bb-6b60-473f-991b-19208337235d service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:47.739802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:47.740559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:47.741338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:47.741649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] PATH_INFO: `/v3/users/8c3ddbedbda24291a35d9c24dbefa3a6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:47.753251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c3ddbedbda24291a35d9c24dbefa3a6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:47.754496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:47.955266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a6800-49fb-4c1e-a72c-df4a5aa73e0f None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:47.955618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a6800-49fb-4c1e-a72c-df4a5aa73e0f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:47.955957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a6800-49fb-4c1e-a72c-df4a5aa73e0f None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:47.958180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 833/1652] 10.222.0.57 () {54 vars in 962 bytes} [Tue Aug 9 18:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:48:47.967663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e3563-d4b4-40f1-85f8-868d215bcabb None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:47.972551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e3563-d4b4-40f1-85f8-868d215bcabb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:47.972551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e3563-d4b4-40f1-85f8-868d215bcabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:47.978985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c3ddbedbda24291a35d9c24dbefa3a6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:48.014366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e90e3563-d4b4-40f1-85f8-868d215bcabb None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:48.015596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e90e3563-d4b4-40f1-85f8-868d215bcabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:48.093026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c3ddbedbda24291a35d9c24dbefa3a6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:48.093645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c3ddbedbda24291a35d9c24dbefa3a6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:48.105875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:48.108273 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c3ddbedbda24291a35d9c24dbefa3a6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:48.135867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 834/1653] 10.222.0.57 () {58 vars in 1059 bytes} [Tue Aug 9 18:48:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:48.135867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a65930-5e5f-4702-8b47-6dc95a341bbf None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:48.135867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a65930-5e5f-4702-8b47-6dc95a341bbf None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:48.135867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a65930-5e5f-4702-8b47-6dc95a341bbf None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:48.139201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 835/1654] 10.222.0.57 () {54 vars in 912 bytes} [Tue Aug 9 18:48:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:48:48.139868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c3ddbedbda24291a35d9c24dbefa3a6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:48.147523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07053cf-8af2-4cd7-9536-0a2ecbd812b9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:48.149624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07053cf-8af2-4cd7-9536-0a2ecbd812b9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:48.150040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07053cf-8af2-4cd7-9536-0a2ecbd812b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:48.155092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 820/1655] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:48:47 2022] DELETE /identity/v3/users/8c3ddbedbda24291a35d9c24dbefa3a6 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:48.177886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c07053cf-8af2-4cd7-9536-0a2ecbd812b9 None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:48.179599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c07053cf-8af2-4cd7-9536-0a2ecbd812b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:48.207660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:48.315341 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 836/1656] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:48:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:48:48.342534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:48.344230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:48.539374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae06ed-abba-4f66-ad34-008b5c2bbb6a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:48.555113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a0335-88ec-4a5e-a2bb-acdb2b7422f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:48.555941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a0335-88ec-4a5e-a2bb-acdb2b7422f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:48.556426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a0335-88ec-4a5e-a2bb-acdb2b7422f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:48.556798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a0335-88ec-4a5e-a2bb-acdb2b7422f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:48.582431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a0335-88ec-4a5e-a2bb-acdb2b7422f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:48.585450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a0335-88ec-4a5e-a2bb-acdb2b7422f6 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:48.621021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 821/1657] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:48.763575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07053cf-8af2-4cd7-9536-0a2ecbd812b9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:48.765684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c07053cf-8af2-4cd7-9536-0a2ecbd812b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:48.939536 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07053cf-8af2-4cd7-9536-0a2ecbd812b9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:48.960895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:48.961561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:48.961873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:48.962146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] PATH_INFO: `/v3/users/cdc9777c9c4c461b849bec17dc82997d` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:48.964411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc9777c9c4c461b849bec17dc82997d)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:48.979665 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:49.279016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc9777c9c4c461b849bec17dc82997d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:49.338365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc9777c9c4c461b849bec17dc82997d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:49.339222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc9777c9c4c461b849bec17dc82997d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:49.347606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc9777c9c4c461b849bec17dc82997d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:49.373001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc9777c9c4c461b849bec17dc82997d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:49.393144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 837/1658] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:48:48 2022] DELETE /identity/v3/users/cdc9777c9c4c461b849bec17dc82997d => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:49.398533 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3e9ec-e5c1-47ec-94ac-6f0767947fd9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:49.398886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3e9ec-e5c1-47ec-94ac-6f0767947fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:49.399223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3e9ec-e5c1-47ec-94ac-6f0767947fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:49.428605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d0b3e9ec-e5c1-47ec-94ac-6f0767947fd9 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:49.432137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d0b3e9ec-e5c1-47ec-94ac-6f0767947fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:49.588332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 822/1659] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:49.644326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:49.645850 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:49.832168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef1da3-e850-43ac-b992-261db167ee45 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:49.854306 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493b688-b8c2-4ebf-8651-81188113c8bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:49.854934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493b688-b8c2-4ebf-8651-81188113c8bb service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:49.855220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493b688-b8c2-4ebf-8651-81188113c8bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:49.855502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493b688-b8c2-4ebf-8651-81188113c8bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:49.879159 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:49.881267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493b688-b8c2-4ebf-8651-81188113c8bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:49.882764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493b688-b8c2-4ebf-8651-81188113c8bb service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:49.912854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 838/1660] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:50.135423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3e9ec-e5c1-47ec-94ac-6f0767947fd9 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:50.136989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d0b3e9ec-e5c1-47ec-94ac-6f0767947fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:50.173313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3e9ec-e5c1-47ec-94ac-6f0767947fd9 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:50.201387 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21719b8c-c767-42e9-98e8-bf7c20d7ff97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:50.209976 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21719b8c-c767-42e9-98e8-bf7c20d7ff97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:50.212499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21719b8c-c767-42e9-98e8-bf7c20d7ff97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:50.212940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21719b8c-c767-42e9-98e8-bf7c20d7ff97 admin admin] PATH_INFO: `/v3/projects/786a9cab00e24528adeb75420779007f` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:50.223521 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21719b8c-c767-42e9-98e8-bf7c20d7ff97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=786a9cab00e24528adeb75420779007f)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:50.229777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21719b8c-c767-42e9-98e8-bf7c20d7ff97 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:50.454916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-21719b8c-c767-42e9-98e8-bf7c20d7ff97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '786a9cab00e24528adeb75420779007f', 'request_id': 'req-21719b8c-c767-42e9-98e8-bf7c20d7ff97'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:50.496223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-21719b8c-c767-42e9-98e8-bf7c20d7ff97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '786a9cab00e24528adeb75420779007f', 'request_id': 'req-21719b8c-c767-42e9-98e8-bf7c20d7ff97'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:50.511852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-21719b8c-c767-42e9-98e8-bf7c20d7ff97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '786a9cab00e24528adeb75420779007f', 'request_id': 'req-21719b8c-c767-42e9-98e8-bf7c20d7ff97'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:50.654700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-21719b8c-c767-42e9-98e8-bf7c20d7ff97 admin admin] The token cache is being invalidate because project 786a9cab00e24528adeb75420779007f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:48:50.655330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-21719b8c-c767-42e9-98e8-bf7c20d7ff97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:50.655906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-21719b8c-c767-42e9-98e8-bf7c20d7ff97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:50.659957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 823/1661] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:48:50 2022] DELETE /identity/v3/projects/786a9cab00e24528adeb75420779007f => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:50.669740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d490f-d14c-40fe-b7b8-1b1af0820228 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:50.670171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d490f-d14c-40fe-b7b8-1b1af0820228 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:50.670295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d490f-d14c-40fe-b7b8-1b1af0820228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:50.755320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6a7d490f-d14c-40fe-b7b8-1b1af0820228 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:50.756795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6a7d490f-d14c-40fe-b7b8-1b1af0820228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:50.961508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 839/1662] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:50.970938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5be7b-40a9-407d-9daa-9a79463b7ac2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:50.971295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5be7b-40a9-407d-9daa-9a79463b7ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:50.971712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5be7b-40a9-407d-9daa-9a79463b7ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:50.994777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:51.036415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d846828-d180-4cc0-8e62-e363a394358b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:51.036872 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d846828-d180-4cc0-8e62-e363a394358b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:51.037377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d846828-d180-4cc0-8e62-e363a394358b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:51.040694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-09a5be7b-40a9-407d-9daa-9a79463b7ac2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:51.044706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-09a5be7b-40a9-407d-9daa-9a79463b7ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:51.067207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 824/1663] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:51.075174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5be7b-40a9-407d-9daa-9a79463b7ac2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:51.099533 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5be7b-40a9-407d-9daa-9a79463b7ac2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:51.125166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53510250-1b50-42ab-86a1-bbfbb9a25daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:51.126337 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53510250-1b50-42ab-86a1-bbfbb9a25daa admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:51.126829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53510250-1b50-42ab-86a1-bbfbb9a25daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:51.127485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53510250-1b50-42ab-86a1-bbfbb9a25daa admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:51.128875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53510250-1b50-42ab-86a1-bbfbb9a25daa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:48:51.131535 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53510250-1b50-42ab-86a1-bbfbb9a25daa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:51.133453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53510250-1b50-42ab-86a1-bbfbb9a25daa admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:51.133584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2d846828-d180-4cc0-8e62-e363a394358b None None] MFA Rules not processed for user `a6e48ee30c61424ba0c82ac4fc5cd06c`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:51.134794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2d846828-d180-4cc0-8e62-e363a394358b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:51.172112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 825/1664] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:48:51.176458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53510250-1b50-42ab-86a1-bbfbb9a25daa admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:51.180442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-53510250-1b50-42ab-86a1-bbfbb9a25daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:51.213145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53510250-1b50-42ab-86a1-bbfbb9a25daa admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:51.229730 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:51.229730 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:51.229730 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:51.229730 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] PATH_INFO: `/v3/projects/ad76f9f6c7b44b8c88c02b65f4ac3d87` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:51.230440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad76f9f6c7b44b8c88c02b65f4ac3d87)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:51.232371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:51.263201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 840/1665] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:51.264405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d846828-d180-4cc0-8e62-e363a394358b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:51.287221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d846828-d180-4cc0-8e62-e363a394358b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:51.301070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4906e-f806-4403-a8b5-910286551384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:51.301736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4906e-f806-4403-a8b5-910286551384 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:51.302266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4906e-f806-4403-a8b5-910286551384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:51.302723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4906e-f806-4403-a8b5-910286551384 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:51.303529 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4906e-f806-4403-a8b5-910286551384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:51.305667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4906e-f806-4403-a8b5-910286551384 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:51.442315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 841/1666] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:51 2022] POST /identity/v3/projects => generated 363 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:51.442315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4906e-f806-4403-a8b5-910286551384 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:51.447915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad76f9f6c7b44b8c88c02b65f4ac3d87', 'request_id': 'req-2d857cf3-0748-4a08-b9a0-50c95e34c168'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:51.449005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1ec4906e-f806-4403-a8b5-910286551384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:51.457692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad76f9f6c7b44b8c88c02b65f4ac3d87', 'request_id': 'req-2d857cf3-0748-4a08-b9a0-50c95e34c168'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:51.472339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad76f9f6c7b44b8c88c02b65f4ac3d87', 'request_id': 'req-2d857cf3-0748-4a08-b9a0-50c95e34c168'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:51.606821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] The token cache is being invalidate because project ad76f9f6c7b44b8c88c02b65f4ac3d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:48:51.606821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:51.607297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:51.610138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 826/1667] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:48:51 2022] DELETE /identity/v3/projects/ad76f9f6c7b44b8c88c02b65f4ac3d87 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:51.613121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:51.616284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:51.632921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4906e-f806-4403-a8b5-910286551384 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:51.648847 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c1101-c853-4517-86aa-7b9dbc60155f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:51.649385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c1101-c853-4517-86aa-7b9dbc60155f service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:51.651968 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c1101-c853-4517-86aa-7b9dbc60155f service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:51.652290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c1101-c853-4517-86aa-7b9dbc60155f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:51.662466 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ff6c1101-c853-4517-86aa-7b9dbc60155f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:51.747094 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d857cf3-0748-4a08-b9a0-50c95e34c168 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:51.780331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:51.833161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c1101-c853-4517-86aa-7b9dbc60155f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:51.838015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c1101-c853-4517-86aa-7b9dbc60155f service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:51.872731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f4f8d-e262-46c9-93f0-2f84abd6592c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:51.873375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f4f8d-e262-46c9-93f0-2f84abd6592c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:51.873716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f4f8d-e262-46c9-93f0-2f84abd6592c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:51.873950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f4f8d-e262-46c9-93f0-2f84abd6592c admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:51.876747 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f4f8d-e262-46c9-93f0-2f84abd6592c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:51.882487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 842/1668] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:51.887619 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c1101-c853-4517-86aa-7b9dbc60155f service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:51.895339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f4f8d-e262-46c9-93f0-2f84abd6592c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:51.895733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ff6c1101-c853-4517-86aa-7b9dbc60155f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:51.916729 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c1101-c853-4517-86aa-7b9dbc60155f service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:51.935122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fb2c9-9819-4833-9ce1-9717b868a883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:51.936111 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fb2c9-9819-4833-9ce1-9717b868a883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:51.936511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fb2c9-9819-4833-9ce1-9717b868a883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:51.936872 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fb2c9-9819-4833-9ce1-9717b868a883 admin admin] PATH_INFO: `/v3/users/edc94bc025714599941e66096fc10acf` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:51.942831 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fb2c9-9819-4833-9ce1-9717b868a883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc94bc025714599941e66096fc10acf)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:51.945501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fb2c9-9819-4833-9ce1-9717b868a883 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:52.059185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 827/1669] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:51 2022] POST /identity/v3/users => generated 282 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:52.062687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f4f8d-e262-46c9-93f0-2f84abd6592c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:52.100030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f4f8d-e262-46c9-93f0-2f84abd6592c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:52.121861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee56fc-0d1b-46be-a8c7-a7395f910668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:52.122538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee56fc-0d1b-46be-a8c7-a7395f910668 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:52.122825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee56fc-0d1b-46be-a8c7-a7395f910668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:52.123205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee56fc-0d1b-46be-a8c7-a7395f910668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:52.124106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee56fc-0d1b-46be-a8c7-a7395f910668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:52.127404 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee56fc-0d1b-46be-a8c7-a7395f910668 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:52.145368 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 828/1670] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:48:52.147354 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee56fc-0d1b-46be-a8c7-a7395f910668 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:52.189005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee56fc-0d1b-46be-a8c7-a7395f910668 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:52.230671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-305fb2c9-9819-4833-9ce1-9717b868a883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc94bc025714599941e66096fc10acf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:52.232460 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9bb8c-a540-4694-8006-d644c7de4fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:52.233149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9bb8c-a540-4694-8006-d644c7de4fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:52.234716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9bb8c-a540-4694-8006-d644c7de4fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:52.235365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9bb8c-a540-4694-8006-d644c7de4fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:52.238288 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9bb8c-a540-4694-8006-d644c7de4fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:52.240407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9bb8c-a540-4694-8006-d644c7de4fc7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:52.295754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 829/1671] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:52.300107 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9bb8c-a540-4694-8006-d644c7de4fc7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:52.312532 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-305fb2c9-9819-4833-9ce1-9717b868a883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc94bc025714599941e66096fc10acf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:52.314176 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-305fb2c9-9819-4833-9ce1-9717b868a883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc94bc025714599941e66096fc10acf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:52.329259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9bb8c-a540-4694-8006-d644c7de4fc7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:52.330049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-305fb2c9-9819-4833-9ce1-9717b868a883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc94bc025714599941e66096fc10acf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:52.341799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ea900-dc61-4683-b2e1-f97a67e63cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:52.342708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ea900-dc61-4683-b2e1-f97a67e63cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:52.343177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ea900-dc61-4683-b2e1-f97a67e63cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:52.343623 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ea900-dc61-4683-b2e1-f97a67e63cf7 admin admin] PATH_INFO: `/v3/projects/98541d0a697a41bc8b1b865a3a5acaf2/users/a28130cce5f641618d2bbb4023469951/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:52.382390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-305fb2c9-9819-4833-9ce1-9717b868a883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc94bc025714599941e66096fc10acf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:52.394477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ea900-dc61-4683-b2e1-f97a67e63cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98541d0a697a41bc8b1b865a3a5acaf2, user_id=a28130cce5f641618d2bbb4023469951, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:52.404973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 843/1672] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:48:51 2022] DELETE /identity/v3/users/edc94bc025714599941e66096fc10acf => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:52.417656 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ea900-dc61-4683-b2e1-f97a67e63cf7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:52.582818 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 830/1673] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:52 2022] PUT /identity/v3/projects/98541d0a697a41bc8b1b865a3a5acaf2/users/a28130cce5f641618d2bbb4023469951/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:52.596128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970d71-fd84-48bd-854e-1120334dfd88 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:52.596418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970d71-fd84-48bd-854e-1120334dfd88 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:52.597243 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970d71-fd84-48bd-854e-1120334dfd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:52.677554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-44970d71-fd84-48bd-854e-1120334dfd88 None None] MFA Rules not processed for user `a28130cce5f641618d2bbb4023469951`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:52.679725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-44970d71-fd84-48bd-854e-1120334dfd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:52.778559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:52.848562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:52.851662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 844/1674] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:52.860414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ea900-dc61-4683-b2e1-f97a67e63cf7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:52.863728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-974ea900-dc61-4683-b2e1-f97a67e63cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:53.033835 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ea900-dc61-4683-b2e1-f97a67e63cf7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:53.049576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4174aa-fc5e-4ff0-b65a-d7d5a6f6196c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:53.052128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4174aa-fc5e-4ff0-b65a-d7d5a6f6196c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:53.053543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4174aa-fc5e-4ff0-b65a-d7d5a6f6196c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:53.056574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4174aa-fc5e-4ff0-b65a-d7d5a6f6196c admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:53.058855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4174aa-fc5e-4ff0-b65a-d7d5a6f6196c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:53.064006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4174aa-fc5e-4ff0-b65a-d7d5a6f6196c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:53.230321 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 831/1675] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:52 2022] POST /identity/v3/projects => generated 367 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:53.235204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970d71-fd84-48bd-854e-1120334dfd88 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:53.259197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970d71-fd84-48bd-854e-1120334dfd88 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:53.406681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace290dc-73a1-4ae5-9789-0f493410fec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:53.407307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace290dc-73a1-4ae5-9789-0f493410fec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:53.407651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace290dc-73a1-4ae5-9789-0f493410fec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:53.412074 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace290dc-73a1-4ae5-9789-0f493410fec2 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:53.417810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace290dc-73a1-4ae5-9789-0f493410fec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:53.420176 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace290dc-73a1-4ae5-9789-0f493410fec2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:53.585602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 845/1676] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:53 2022] POST /identity/v3/users => generated 290 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:53.590016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4174aa-fc5e-4ff0-b65a-d7d5a6f6196c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:53.609193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4174aa-fc5e-4ff0-b65a-d7d5a6f6196c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:53.625728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85243f2b-3f31-4b0d-8c31-c12edaddc188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:53.627232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85243f2b-3f31-4b0d-8c31-c12edaddc188 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:53.627517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85243f2b-3f31-4b0d-8c31-c12edaddc188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:53.631395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85243f2b-3f31-4b0d-8c31-c12edaddc188 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:53.632226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85243f2b-3f31-4b0d-8c31-c12edaddc188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:53.639404 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85243f2b-3f31-4b0d-8c31-c12edaddc188 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:53.674315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 832/1677] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:53.679503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace290dc-73a1-4ae5-9789-0f493410fec2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:53.711304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace290dc-73a1-4ae5-9789-0f493410fec2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:53.724653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb9d6a-0f04-4cb7-b63f-d529ee69f77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:53.727261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb9d6a-0f04-4cb7-b63f-d529ee69f77e admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:53.727635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb9d6a-0f04-4cb7-b63f-d529ee69f77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:53.728986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb9d6a-0f04-4cb7-b63f-d529ee69f77e admin admin] PATH_INFO: `/v3/projects/90161cca676f4458937680f28d1f4a1c/users/3c7e13e7bdd24df38072faede964f8f9/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:53.778740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb9d6a-0f04-4cb7-b63f-d529ee69f77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=90161cca676f4458937680f28d1f4a1c, user_id=3c7e13e7bdd24df38072faede964f8f9, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:53.781637 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb9d6a-0f04-4cb7-b63f-d529ee69f77e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:53.867555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 846/1678] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:53 2022] PUT /identity/v3/projects/90161cca676f4458937680f28d1f4a1c/users/3c7e13e7bdd24df38072faede964f8f9/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:48:53.871547 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85243f2b-3f31-4b0d-8c31-c12edaddc188 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:53.888703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85243f2b-3f31-4b0d-8c31-c12edaddc188 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:54.039438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fd850-dbb2-4604-9637-6d0280461e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:54.040260 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fd850-dbb2-4604-9637-6d0280461e51 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:54.040738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fd850-dbb2-4604-9637-6d0280461e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:54.041193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fd850-dbb2-4604-9637-6d0280461e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:54.042033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fd850-dbb2-4604-9637-6d0280461e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:54.043705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fd850-dbb2-4604-9637-6d0280461e51 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:54.059483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 833/1679] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:54.064409 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb9d6a-0f04-4cb7-b63f-d529ee69f77e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:54.081724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb9d6a-0f04-4cb7-b63f-d529ee69f77e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:54.101032 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5295b-0da4-4d9c-bec7-b2d5cd587eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:54.103988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5295b-0da4-4d9c-bec7-b2d5cd587eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:54.104283 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5295b-0da4-4d9c-bec7-b2d5cd587eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:54.104565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5295b-0da4-4d9c-bec7-b2d5cd587eb9 admin admin] PATH_INFO: `/v3/projects/90161cca676f4458937680f28d1f4a1c/users/3c7e13e7bdd24df38072faede964f8f9/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:54.111952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5295b-0da4-4d9c-bec7-b2d5cd587eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90161cca676f4458937680f28d1f4a1c, user_id=3c7e13e7bdd24df38072faede964f8f9, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:54.125917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5295b-0da4-4d9c-bec7-b2d5cd587eb9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:54.179092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 847/1680] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:54 2022] PUT /identity/v3/projects/90161cca676f4458937680f28d1f4a1c/users/3c7e13e7bdd24df38072faede964f8f9/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:54.180362 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91134319-3c44-41f3-9143-b7842ba6aa34 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:54.180657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91134319-3c44-41f3-9143-b7842ba6aa34 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:54.180939 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91134319-3c44-41f3-9143-b7842ba6aa34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:54.305383 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-91134319-3c44-41f3-9143-b7842ba6aa34 None None] MFA Rules not processed for user `3c7e13e7bdd24df38072faede964f8f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:54.306832 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-91134319-3c44-41f3-9143-b7842ba6aa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:54.345286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:54.481844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 834/1681] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:54.582938 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5295b-0da4-4d9c-bec7-b2d5cd587eb9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:54.588698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-62b5295b-0da4-4d9c-bec7-b2d5cd587eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:54.700913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5295b-0da4-4d9c-bec7-b2d5cd587eb9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:54.710386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355b48b-f835-4c6f-b9e4-5ddefe9c01ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:54.712918 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355b48b-f835-4c6f-b9e4-5ddefe9c01ee service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:54.713424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355b48b-f835-4c6f-b9e4-5ddefe9c01ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:54.715321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355b48b-f835-4c6f-b9e4-5ddefe9c01ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:54.734776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355b48b-f835-4c6f-b9e4-5ddefe9c01ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:54.736909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355b48b-f835-4c6f-b9e4-5ddefe9c01ee service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:54.756379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 848/1682] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:54.838099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91134319-3c44-41f3-9143-b7842ba6aa34 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:54.840762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-91134319-3c44-41f3-9143-b7842ba6aa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:55.014817 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91134319-3c44-41f3-9143-b7842ba6aa34 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:55.059700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74844b3-cea1-4b68-b255-83e99e6f7687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:55.062549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74844b3-cea1-4b68-b255-83e99e6f7687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:55.062986 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74844b3-cea1-4b68-b255-83e99e6f7687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:55.063905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74844b3-cea1-4b68-b255-83e99e6f7687 admin admin] PATH_INFO: `/v3/projects/c5fa2cda24ed4953876fe8af1fa69a4f` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:55.077105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74844b3-cea1-4b68-b255-83e99e6f7687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5fa2cda24ed4953876fe8af1fa69a4f)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:55.081959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74844b3-cea1-4b68-b255-83e99e6f7687 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:55.082165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9c38-8391-4ef8-84a6-3582fe6b3879 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:55.083915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9c38-8391-4ef8-84a6-3582fe6b3879 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:55.084476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9c38-8391-4ef8-84a6-3582fe6b3879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:55.145623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-52be9c38-8391-4ef8-84a6-3582fe6b3879 None None] MFA Rules not processed for user `7fde48fb289b4c7591658e27ce5ff55e`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:55.147205 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-52be9c38-8391-4ef8-84a6-3582fe6b3879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:55.197883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:55.277639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d74844b3-cea1-4b68-b255-83e99e6f7687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5fa2cda24ed4953876fe8af1fa69a4f', 'request_id': 'req-d74844b3-cea1-4b68-b255-83e99e6f7687'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:55.295971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d74844b3-cea1-4b68-b255-83e99e6f7687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5fa2cda24ed4953876fe8af1fa69a4f', 'request_id': 'req-d74844b3-cea1-4b68-b255-83e99e6f7687'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:55.319803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d74844b3-cea1-4b68-b255-83e99e6f7687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5fa2cda24ed4953876fe8af1fa69a4f', 'request_id': 'req-d74844b3-cea1-4b68-b255-83e99e6f7687'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:55.338782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 849/1683] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:55.341149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd674f1-4aa0-45ee-bf6d-4cdde41c00ca None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:55.341438 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd674f1-4aa0-45ee-bf6d-4cdde41c00ca None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:55.341744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd674f1-4aa0-45ee-bf6d-4cdde41c00ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:55.386129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0cd674f1-4aa0-45ee-bf6d-4cdde41c00ca None None] MFA Rules not processed for user `7fde48fb289b4c7591658e27ce5ff55e`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:55.386129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0cd674f1-4aa0-45ee-bf6d-4cdde41c00ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:55.469936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d74844b3-cea1-4b68-b255-83e99e6f7687 admin admin] The token cache is being invalidate because project c5fa2cda24ed4953876fe8af1fa69a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:48:55.470194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d74844b3-cea1-4b68-b255-83e99e6f7687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:55.473681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d74844b3-cea1-4b68-b255-83e99e6f7687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:55.478613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 835/1684] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:48:54 2022] DELETE /identity/v3/projects/c5fa2cda24ed4953876fe8af1fa69a4f => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:55.483239 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a8034-0eae-4d7b-a390-413a2fcdf2a9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:55.483534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a8034-0eae-4d7b-a390-413a2fcdf2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:55.483811 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a8034-0eae-4d7b-a390-413a2fcdf2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:55.514555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d47a8034-0eae-4d7b-a390-413a2fcdf2a9 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:55.514916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d47a8034-0eae-4d7b-a390-413a2fcdf2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:55.516285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 850/1685] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:55.534027 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd674f1-4aa0-45ee-bf6d-4cdde41c00ca None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:55.536002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0cd674f1-4aa0-45ee-bf6d-4cdde41c00ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:55.634718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:55.689689 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd674f1-4aa0-45ee-bf6d-4cdde41c00ca None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:55.701004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 836/1686] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:48:55.703160 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfc4c4-670a-4fc0-9217-0c57a8140f73 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:55.703527 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfc4c4-670a-4fc0-9217-0c57a8140f73 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:55.704005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfc4c4-670a-4fc0-9217-0c57a8140f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:55.704398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04e561-3212-4b75-87c9-234495564e25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:55.705449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04e561-3212-4b75-87c9-234495564e25 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:55.705973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04e561-3212-4b75-87c9-234495564e25 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:55.706426 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04e561-3212-4b75-87c9-234495564e25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:55.710740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cc04e561-3212-4b75-87c9-234495564e25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:55.735205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-68cfc4c4-670a-4fc0-9217-0c57a8140f73 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:55.737916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-68cfc4c4-670a-4fc0-9217-0c57a8140f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:55.738980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04e561-3212-4b75-87c9-234495564e25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:55.740955 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04e561-3212-4b75-87c9-234495564e25 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:55.769894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 851/1687] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:48:55.773685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 837/1688] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:55.778655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04e561-3212-4b75-87c9-234495564e25 service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:55.810905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04e561-3212-4b75-87c9-234495564e25 service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:55.826490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a956b53-3972-4f0f-bcdb-930f21b8e2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:55.828149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a956b53-3972-4f0f-bcdb-930f21b8e2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:55.831448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a956b53-3972-4f0f-bcdb-930f21b8e2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:55.832273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a956b53-3972-4f0f-bcdb-930f21b8e2d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:55.834624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a956b53-3972-4f0f-bcdb-930f21b8e2d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:48:55.835311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a956b53-3972-4f0f-bcdb-930f21b8e2d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:55.841139 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a956b53-3972-4f0f-bcdb-930f21b8e2d2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:55.904376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfc4c4-670a-4fc0-9217-0c57a8140f73 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:55.905376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 852/1689] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:48:55.915748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfc4c4-670a-4fc0-9217-0c57a8140f73 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:55.946016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301df6e-78d2-4df3-a549-c01f1f0c096d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:55.946016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301df6e-78d2-4df3-a549-c01f1f0c096d admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:55.946016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301df6e-78d2-4df3-a549-c01f1f0c096d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:55.946016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301df6e-78d2-4df3-a549-c01f1f0c096d admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:55.946319 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301df6e-78d2-4df3-a549-c01f1f0c096d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:55.964447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301df6e-78d2-4df3-a549-c01f1f0c096d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:56.122374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 838/1690] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:55 2022] POST /identity/v3/projects => generated 375 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:56.126646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a956b53-3972-4f0f-bcdb-930f21b8e2d2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:56.141056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a956b53-3972-4f0f-bcdb-930f21b8e2d2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:56.345487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76479b-a6a3-4aa3-b2d7-469077aa3195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:56.346885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76479b-a6a3-4aa3-b2d7-469077aa3195 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:56.347387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76479b-a6a3-4aa3-b2d7-469077aa3195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:56.348081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76479b-a6a3-4aa3-b2d7-469077aa3195 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:56.349687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76479b-a6a3-4aa3-b2d7-469077aa3195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:56.355456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76479b-a6a3-4aa3-b2d7-469077aa3195 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:56.514272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 853/1691] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:56 2022] POST /identity/v3/users => generated 288 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:56.516830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301df6e-78d2-4df3-a549-c01f1f0c096d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:56.535798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301df6e-78d2-4df3-a549-c01f1f0c096d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:56.549412 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4d9c4-49c9-46e2-8136-452397482615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:56.553784 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4d9c4-49c9-46e2-8136-452397482615 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:56.553784 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4d9c4-49c9-46e2-8136-452397482615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:56.555233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4d9c4-49c9-46e2-8136-452397482615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:56.555233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4d9c4-49c9-46e2-8136-452397482615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:56.566715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4d9c4-49c9-46e2-8136-452397482615 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:56.598660 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 839/1692] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:56.603052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76479b-a6a3-4aa3-b2d7-469077aa3195 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:56.619546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76479b-a6a3-4aa3-b2d7-469077aa3195 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:56.637719 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c67ca0-9a20-46e3-aae1-2bf6e825de0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:56.638583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c67ca0-9a20-46e3-aae1-2bf6e825de0a admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:56.638744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c67ca0-9a20-46e3-aae1-2bf6e825de0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:56.639106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c67ca0-9a20-46e3-aae1-2bf6e825de0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:56.639838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c67ca0-9a20-46e3-aae1-2bf6e825de0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:56.641567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c67ca0-9a20-46e3-aae1-2bf6e825de0a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:56.658503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 854/1693] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:56.664312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4d9c4-49c9-46e2-8136-452397482615 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:56.679761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4d9c4-49c9-46e2-8136-452397482615 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:56.700634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13b6b2-0def-4f88-a303-bf05a6c480db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:56.700634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13b6b2-0def-4f88-a303-bf05a6c480db admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:56.700634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13b6b2-0def-4f88-a303-bf05a6c480db admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:56.700634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13b6b2-0def-4f88-a303-bf05a6c480db admin admin] PATH_INFO: `/v3/projects/b86f28e15c8f42f4bc89e28fea05cead/users/6c6a63d12bce4014898d450819f10ef3/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:56.722085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:56.740544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13b6b2-0def-4f88-a303-bf05a6c480db admin admin] RBAC: Authorizing `identity:create_grant(project_id=b86f28e15c8f42f4bc89e28fea05cead, user_id=6c6a63d12bce4014898d450819f10ef3, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:56.743090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13b6b2-0def-4f88-a303-bf05a6c480db admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:56.847323 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 840/1694] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:56 2022] PUT /identity/v3/projects/b86f28e15c8f42f4bc89e28fea05cead/users/6c6a63d12bce4014898d450819f10ef3/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:56.857492 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1aed1-d081-402a-8cbf-133b424c2e2b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:56.859567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1aed1-d081-402a-8cbf-133b424c2e2b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:56.859865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1aed1-d081-402a-8cbf-133b424c2e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:56.869783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e27bb3-8d4a-449e-a457-74390f23b555 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:56.870206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e27bb3-8d4a-449e-a457-74390f23b555 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:56.870487 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e27bb3-8d4a-449e-a457-74390f23b555 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:56.872236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 841/1695] 10.222.0.57 () {54 vars in 912 bytes} [Tue Aug 9 18:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:48:56.877985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b0257-1913-4659-9462-636974f4d410 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:56.878173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b0257-1913-4659-9462-636974f4d410 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:56.878471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b0257-1913-4659-9462-636974f4d410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:56.977463 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0c1b0257-1913-4659-9462-636974f4d410 None None] MFA Rules not processed for user `88a7cdd610f74330a5da69c0de8472db`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:56.977463 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0c1b0257-1913-4659-9462-636974f4d410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:56.986095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-24a1aed1-d081-402a-8cbf-133b424c2e2b None None] MFA Rules not processed for user `6c6a63d12bce4014898d450819f10ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:56.989632 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-24a1aed1-d081-402a-8cbf-133b424c2e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:57.107050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 842/1696] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:48:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:48:57.108484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f5dd8-3f77-48cf-aebc-77fc460131b8 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:57.108781 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f5dd8-3f77-48cf-aebc-77fc460131b8 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:57.109054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f5dd8-3f77-48cf-aebc-77fc460131b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:57.188698 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-249f5dd8-3f77-48cf-aebc-77fc460131b8 None None] MFA Rules not processed for user `ef8af9fd9e844ca4be10caa880357a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:57.191939 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-249f5dd8-3f77-48cf-aebc-77fc460131b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:57.233272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 855/1697] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:57.238336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1aed1-d081-402a-8cbf-133b424c2e2b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:57.241461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-24a1aed1-d081-402a-8cbf-133b424c2e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:57.304622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1aed1-d081-402a-8cbf-133b424c2e2b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:57.319856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60c7f2-b31b-4290-b31c-8af12ab3fd87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:57.324713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60c7f2-b31b-4290-b31c-8af12ab3fd87 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:57.327093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60c7f2-b31b-4290-b31c-8af12ab3fd87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:57.328154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60c7f2-b31b-4290-b31c-8af12ab3fd87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:57.362761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60c7f2-b31b-4290-b31c-8af12ab3fd87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:57.368438 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60c7f2-b31b-4290-b31c-8af12ab3fd87 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:57.381991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 843/1698] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:57.383284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f5dd8-3f77-48cf-aebc-77fc460131b8 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:57.386724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-249f5dd8-3f77-48cf-aebc-77fc460131b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:57.407184 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 856/1699] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:57.408148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60c7f2-b31b-4290-b31c-8af12ab3fd87 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:57.530778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f5dd8-3f77-48cf-aebc-77fc460131b8 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:57.541780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:57.544658 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:57.545057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:57.546388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] PATH_INFO: `/v3/users/bfc90a2b0c054c93afdf725381859dd9` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:57.548474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc90a2b0c054c93afdf725381859dd9)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:57.552919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:57.629736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60c7f2-b31b-4290-b31c-8af12ab3fd87 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:57.645825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db447fa5-558f-450c-b91c-f766273fe056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:57.649650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db447fa5-558f-450c-b91c-f766273fe056 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:57.649977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db447fa5-558f-450c-b91c-f766273fe056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:57.650261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db447fa5-558f-450c-b91c-f766273fe056 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:57.650921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db447fa5-558f-450c-b91c-f766273fe056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:57.657586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db447fa5-558f-450c-b91c-f766273fe056 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:57.848630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 857/1700] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:48:57 2022] POST /identity/v3/projects => generated 375 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:57.849668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db447fa5-558f-450c-b91c-f766273fe056 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:57.852033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-db447fa5-558f-450c-b91c-f766273fe056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:57.860465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc90a2b0c054c93afdf725381859dd9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:57.947103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:57.950617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc90a2b0c054c93afdf725381859dd9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:57.951330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc90a2b0c054c93afdf725381859dd9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:57.968725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc90a2b0c054c93afdf725381859dd9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:58.000415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc90a2b0c054c93afdf725381859dd9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:48:58.018462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 844/1701] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:48:57 2022] DELETE /identity/v3/users/bfc90a2b0c054c93afdf725381859dd9 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:58.022404 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:58.027291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:58.028549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db447fa5-558f-450c-b91c-f766273fe056 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:58.112087 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56c722-123d-4f2f-b2d5-e5dd2bd3246c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:58.113036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56c722-123d-4f2f-b2d5-e5dd2bd3246c service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:58.113458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90450d85-7cc3-41c2-832f-81c5964d7678 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:58.113583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56c722-123d-4f2f-b2d5-e5dd2bd3246c service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:58.113967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56c722-123d-4f2f-b2d5-e5dd2bd3246c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:58.116848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2a56c722-123d-4f2f-b2d5-e5dd2bd3246c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:58.138855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da41531-3c2b-407f-adfb-bd950ca3dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:58.140244 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da41531-3c2b-407f-adfb-bd950ca3dcd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:58.153281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da41531-3c2b-407f-adfb-bd950ca3dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:58.153742 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da41531-3c2b-407f-adfb-bd950ca3dcd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:58.154524 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da41531-3c2b-407f-adfb-bd950ca3dcd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:58.165574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da41531-3c2b-407f-adfb-bd950ca3dcd8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:58.178781 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:58.218114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56c722-123d-4f2f-b2d5-e5dd2bd3246c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:58.220284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56c722-123d-4f2f-b2d5-e5dd2bd3246c service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:58.245096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 858/1702] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:48:58.248752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6002134-824c-4fcc-83a8-0176e79d2b2f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:58.249044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6002134-824c-4fcc-83a8-0176e79d2b2f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:58.249328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6002134-824c-4fcc-83a8-0176e79d2b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:58.314732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a6002134-824c-4fcc-83a8-0176e79d2b2f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:58.318706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 845/1703] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:48:58 2022] POST /identity/v3/users => generated 294 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:48:58.321766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a6002134-824c-4fcc-83a8-0176e79d2b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:58.322754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da41531-3c2b-407f-adfb-bd950ca3dcd8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:58.335144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 859/1704] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:58.347858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6002134-824c-4fcc-83a8-0176e79d2b2f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:58.354074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da41531-3c2b-407f-adfb-bd950ca3dcd8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:58.355089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a6002134-824c-4fcc-83a8-0176e79d2b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:58.369819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8e224-0ca5-4d47-8b35-d27548e5f11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:58.370705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8e224-0ca5-4d47-8b35-d27548e5f11e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:58.371087 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8e224-0ca5-4d47-8b35-d27548e5f11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:58.371898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8e224-0ca5-4d47-8b35-d27548e5f11e admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:58.372574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8e224-0ca5-4d47-8b35-d27548e5f11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:58.374850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8e224-0ca5-4d47-8b35-d27548e5f11e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:58.410939 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 846/1705] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:48:58.412096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8e224-0ca5-4d47-8b35-d27548e5f11e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:58.432965 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8e224-0ca5-4d47-8b35-d27548e5f11e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:58.445325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a5678-bf4f-4b47-bf87-44b172c44208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:58.446099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5678-bf4f-4b47-bf87-44b172c44208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:58.446376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5678-bf4f-4b47-bf87-44b172c44208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:58.446638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5678-bf4f-4b47-bf87-44b172c44208 admin admin] PATH_INFO: `/v3/projects/618d186d70a744c1bf55f7a764d04623/users/27c215f0741d49c28f9c0c5b08d2c1cf/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:58.467168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a5678-bf4f-4b47-bf87-44b172c44208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=618d186d70a744c1bf55f7a764d04623, user_id=27c215f0741d49c28f9c0c5b08d2c1cf, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:58.477612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a5678-bf4f-4b47-bf87-44b172c44208 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:58.528462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6002134-824c-4fcc-83a8-0176e79d2b2f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:58.544358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ed652-beb2-4467-878e-79716692e5c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:58.546109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ed652-beb2-4467-878e-79716692e5c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:58.546391 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ed652-beb2-4467-878e-79716692e5c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:58.546665 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ed652-beb2-4467-878e-79716692e5c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:58.578894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 847/1706] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:58 2022] PUT /identity/v3/projects/618d186d70a744c1bf55f7a764d04623/users/27c215f0741d49c28f9c0c5b08d2c1cf/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:58.582861 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ed652-beb2-4467-878e-79716692e5c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:58.583693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a5678-bf4f-4b47-bf87-44b172c44208 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:58.584536 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ed652-beb2-4467-878e-79716692e5c5 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:58.606671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 860/1707] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:48:58.638585 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17dfca-1978-4f53-bbc5-087eef42bb9d None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:58.639174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17dfca-1978-4f53-bbc5-087eef42bb9d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:58.639801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17dfca-1978-4f53-bbc5-087eef42bb9d None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:58.642253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 861/1708] 10.222.0.57 () {54 vars in 961 bytes} [Tue Aug 9 18:48:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:48:58.647020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90630181-2a79-40ab-8ae6-132f3c8dad62 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:58.647363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90630181-2a79-40ab-8ae6-132f3c8dad62 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:58.652402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90630181-2a79-40ab-8ae6-132f3c8dad62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:58.703121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-90630181-2a79-40ab-8ae6-132f3c8dad62 None None] MFA Rules not processed for user `5e0aa6b215f646c280af245a89ccf849`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:58.707076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-90630181-2a79-40ab-8ae6-132f3c8dad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:58.806245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a5678-bf4f-4b47-bf87-44b172c44208 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:58.835589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c418f-0b2c-4adb-92a2-4c0dab94ae6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:58.837078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c418f-0b2c-4adb-92a2-4c0dab94ae6d admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:58.837697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c418f-0b2c-4adb-92a2-4c0dab94ae6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:58.838267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c418f-0b2c-4adb-92a2-4c0dab94ae6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:58.839206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c418f-0b2c-4adb-92a2-4c0dab94ae6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:58.841608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c418f-0b2c-4adb-92a2-4c0dab94ae6d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:58.859563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 848/1709] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:48:58 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:48:58.860978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c418f-0b2c-4adb-92a2-4c0dab94ae6d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:58.883468 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c418f-0b2c-4adb-92a2-4c0dab94ae6d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:58.889612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 862/1710] 10.222.0.57 () {58 vars in 1058 bytes} [Tue Aug 9 18:48:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:48:58.896477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393ef74-69f2-4e25-b10d-5d13e238578f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:58.897300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393ef74-69f2-4e25-b10d-5d13e238578f admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:58.897612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393ef74-69f2-4e25-b10d-5d13e238578f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:58.897897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393ef74-69f2-4e25-b10d-5d13e238578f admin admin] PATH_INFO: `/v3/projects/618d186d70a744c1bf55f7a764d04623/users/27c215f0741d49c28f9c0c5b08d2c1cf/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:58.898335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90630181-2a79-40ab-8ae6-132f3c8dad62 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:58.904448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393ef74-69f2-4e25-b10d-5d13e238578f admin admin] RBAC: Authorizing `identity:create_grant(project_id=618d186d70a744c1bf55f7a764d04623, user_id=27c215f0741d49c28f9c0c5b08d2c1cf, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:58.917201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90630181-2a79-40ab-8ae6-132f3c8dad62 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:58.918464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393ef74-69f2-4e25-b10d-5d13e238578f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:58.932541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcb235-7998-47ff-8dff-1091d590f0da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0aa6b215f646c280af245a89ccf849', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:58.933176 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcb235-7998-47ff-8dff-1091d590f0da service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:58.934110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcb235-7998-47ff-8dff-1091d590f0da service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:58.934519 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcb235-7998-47ff-8dff-1091d590f0da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:58.941070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d9fcb235-7998-47ff-8dff-1091d590f0da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:58.961271 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 849/1711] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:48:58 2022] PUT /identity/v3/projects/618d186d70a744c1bf55f7a764d04623/users/27c215f0741d49c28f9c0c5b08d2c1cf/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:48:58.977015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcb235-7998-47ff-8dff-1091d590f0da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:58.977559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284796a-2f6d-40c0-9eb3-9788e03615e3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:58.977836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284796a-2f6d-40c0-9eb3-9788e03615e3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:58.978110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284796a-2f6d-40c0-9eb3-9788e03615e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:58.979398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcb235-7998-47ff-8dff-1091d590f0da service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:48:59.051793 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 863/1712] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:48:59.071144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3284796a-2f6d-40c0-9eb3-9788e03615e3 None None] MFA Rules not processed for user `27c215f0741d49c28f9c0c5b08d2c1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:48:59.074165 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3284796a-2f6d-40c0-9eb3-9788e03615e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:59.381975 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 850/1713] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:48:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:48:59.407442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcb235-7998-47ff-8dff-1091d590f0da service placement] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:48:59.410539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d9fcb235-7998-47ff-8dff-1091d590f0da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:59.491435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:59.649855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcb235-7998-47ff-8dff-1091d590f0da service placement] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:48:59.786281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878b033-4589-4434-a202-fdefefe3eff0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:48:59.787873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878b033-4589-4434-a202-fdefefe3eff0 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:48:59.787873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878b033-4589-4434-a202-fdefefe3eff0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:48:59.787873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878b033-4589-4434-a202-fdefefe3eff0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:48:59.798446 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b878b033-4589-4434-a202-fdefefe3eff0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:48:59.887659 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:48:59.995073 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878b033-4589-4434-a202-fdefefe3eff0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:48:59.996835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878b033-4589-4434-a202-fdefefe3eff0 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:00.022962 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 864/1714] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:48:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:00.149983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284796a-2f6d-40c0-9eb3-9788e03615e3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:00.151464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3284796a-2f6d-40c0-9eb3-9788e03615e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:00.304745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:00.359685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284796a-2f6d-40c0-9eb3-9788e03615e3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:00.370464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb552b61-1556-444a-acb4-9f697114ebb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:00.371323 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb552b61-1556-444a-acb4-9f697114ebb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:00.371779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb552b61-1556-444a-acb4-9f697114ebb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:00.372227 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb552b61-1556-444a-acb4-9f697114ebb0 admin admin] PATH_INFO: `/v3/projects/0cd0e883aec7498ca54ea52fa547edde` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:00.374294 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb552b61-1556-444a-acb4-9f697114ebb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd0e883aec7498ca54ea52fa547edde)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:00.376703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb552b61-1556-444a-acb4-9f697114ebb0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:00.589110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-eb552b61-1556-444a-acb4-9f697114ebb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd0e883aec7498ca54ea52fa547edde', 'request_id': 'req-eb552b61-1556-444a-acb4-9f697114ebb0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:00.614001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-eb552b61-1556-444a-acb4-9f697114ebb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd0e883aec7498ca54ea52fa547edde', 'request_id': 'req-eb552b61-1556-444a-acb4-9f697114ebb0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:00.625872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-eb552b61-1556-444a-acb4-9f697114ebb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd0e883aec7498ca54ea52fa547edde', 'request_id': 'req-eb552b61-1556-444a-acb4-9f697114ebb0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:00.707914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca783750-318a-4b2a-978b-0e4204e30712 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:00.708490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca783750-318a-4b2a-978b-0e4204e30712 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:00.708490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca783750-318a-4b2a-978b-0e4204e30712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:00.723558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-eb552b61-1556-444a-acb4-9f697114ebb0 admin admin] The token cache is being invalidate because project 0cd0e883aec7498ca54ea52fa547edde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:00.724058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-eb552b61-1556-444a-acb4-9f697114ebb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:00.725439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-eb552b61-1556-444a-acb4-9f697114ebb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:00.729067 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 851/1715] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:00 2022] DELETE /identity/v3/projects/0cd0e883aec7498ca54ea52fa547edde => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:00.734675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62435ba7-b50e-48b6-ab60-c930fdbb1a0b None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:00.741014 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62435ba7-b50e-48b6-ab60-c930fdbb1a0b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:00.741307 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62435ba7-b50e-48b6-ab60-c930fdbb1a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:00.811983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ca783750-318a-4b2a-978b-0e4204e30712 None None] MFA Rules not processed for user `04da0f4276d6457c889ecb3006842e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:00.816281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ca783750-318a-4b2a-978b-0e4204e30712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:00.826799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-62435ba7-b50e-48b6-ab60-c930fdbb1a0b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:00.830477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-62435ba7-b50e-48b6-ab60-c930fdbb1a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:00.970603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 852/1716] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:00.979524 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a23cd-545c-4423-842c-7c03c1d6e58f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:00.980155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a23cd-545c-4423-842c-7c03c1d6e58f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:00.980800 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a23cd-545c-4423-842c-7c03c1d6e58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:00.985410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 865/1717] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:01.000276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cf94c-9246-4fa2-8440-b2d980bb01bf None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:01.000757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cf94c-9246-4fa2-8440-b2d980bb01bf None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:01.001262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cf94c-9246-4fa2-8440-b2d980bb01bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:01.046653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-805a23cd-545c-4423-842c-7c03c1d6e58f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:01.049100 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-805a23cd-545c-4423-842c-7c03c1d6e58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:01.051308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6c2cf94c-9246-4fa2-8440-b2d980bb01bf None None] MFA Rules not processed for user `04da0f4276d6457c889ecb3006842e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:01.058178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6c2cf94c-9246-4fa2-8440-b2d980bb01bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:01.071764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 853/1718] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:01.078507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 866/1719] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:01.084571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a23cd-545c-4423-842c-7c03c1d6e58f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:01.092263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cf94c-9246-4fa2-8440-b2d980bb01bf None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:01.094003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6c2cf94c-9246-4fa2-8440-b2d980bb01bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:01.102259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a23cd-545c-4423-842c-7c03c1d6e58f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:01.114205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66cc09-ab66-4882-8ce7-ccc663d9f094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:01.115018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66cc09-ab66-4882-8ce7-ccc663d9f094 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:01.117114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66cc09-ab66-4882-8ce7-ccc663d9f094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:01.118035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66cc09-ab66-4882-8ce7-ccc663d9f094 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:01.118877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66cc09-ab66-4882-8ce7-ccc663d9f094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:01.119888 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66cc09-ab66-4882-8ce7-ccc663d9f094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:01.123435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66cc09-ab66-4882-8ce7-ccc663d9f094 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:01.150554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 854/1720] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:01.160317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66cc09-ab66-4882-8ce7-ccc663d9f094 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:01.178779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66cc09-ab66-4882-8ce7-ccc663d9f094 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:01.190205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2955e08-f755-401c-8e27-d61082a46d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:01.191389 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2955e08-f755-401c-8e27-d61082a46d7c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:01.191897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2955e08-f755-401c-8e27-d61082a46d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:01.192373 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2955e08-f755-401c-8e27-d61082a46d7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:01.193512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2955e08-f755-401c-8e27-d61082a46d7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:01.195746 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2955e08-f755-401c-8e27-d61082a46d7c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:01.319242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cf94c-9246-4fa2-8440-b2d980bb01bf None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:01.325993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c38fc-f601-490f-96ec-bcb6d7be339e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:01.326809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c38fc-f601-490f-96ec-bcb6d7be339e service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:01.327242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c38fc-f601-490f-96ec-bcb6d7be339e service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:01.327663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c38fc-f601-490f-96ec-bcb6d7be339e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:01.339145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c38fc-f601-490f-96ec-bcb6d7be339e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:01.340963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c38fc-f601-490f-96ec-bcb6d7be339e service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:01.352674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 867/1721] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:01.354890 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a58bd5-0d3a-4aa0-ae73-9f74adc9a2c9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:01.355396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a58bd5-0d3a-4aa0-ae73-9f74adc9a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:01.355736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a58bd5-0d3a-4aa0-ae73-9f74adc9a2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:01.383746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-18a58bd5-0d3a-4aa0-ae73-9f74adc9a2c9 None None] MFA Rules not processed for user `a6e48ee30c61424ba0c82ac4fc5cd06c`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:01.385010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-18a58bd5-0d3a-4aa0-ae73-9f74adc9a2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:01.424513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 855/1722] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:01 2022] POST /identity/v3/projects => generated 381 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:01.426963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2955e08-f755-401c-8e27-d61082a46d7c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:01.434494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b2955e08-f755-401c-8e27-d61082a46d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:01.522909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 868/1723] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:01.524194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a58bd5-0d3a-4aa0-ae73-9f74adc9a2c9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:01.574761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a58bd5-0d3a-4aa0-ae73-9f74adc9a2c9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:01.581690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:01.657422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2955e08-f755-401c-8e27-d61082a46d7c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:01.670962 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c83c6-fa03-4bcb-a8c4-d7f16e754f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:01.672578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c83c6-fa03-4bcb-a8c4-d7f16e754f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:01.673040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c83c6-fa03-4bcb-a8c4-d7f16e754f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:01.674759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c83c6-fa03-4bcb-a8c4-d7f16e754f51 admin admin] PATH_INFO: `/v3/users/a28130cce5f641618d2bbb4023469951` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:01.676992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c83c6-fa03-4bcb-a8c4-d7f16e754f51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a28130cce5f641618d2bbb4023469951)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:01.680748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c83c6-fa03-4bcb-a8c4-d7f16e754f51 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:01.753550 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30eb3ab-f807-4a55-85c4-0a550db5cbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:01.754171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30eb3ab-f807-4a55-85c4-0a550db5cbcf admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:01.755181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30eb3ab-f807-4a55-85c4-0a550db5cbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:01.755494 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30eb3ab-f807-4a55-85c4-0a550db5cbcf admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:01.757915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30eb3ab-f807-4a55-85c4-0a550db5cbcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:01.762287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30eb3ab-f807-4a55-85c4-0a550db5cbcf admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:01.957744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 869/1724] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:01 2022] POST /identity/v3/users => generated 291 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:01.961316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30eb3ab-f807-4a55-85c4-0a550db5cbcf admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:01.990431 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30eb3ab-f807-4a55-85c4-0a550db5cbcf admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:02.008006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46132db6-836f-4ee4-8066-586e3329a7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:02.008911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46132db6-836f-4ee4-8066-586e3329a7da admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:02.009399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46132db6-836f-4ee4-8066-586e3329a7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:02.009898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46132db6-836f-4ee4-8066-586e3329a7da admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:02.010757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46132db6-836f-4ee4-8066-586e3329a7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:02.013400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46132db6-836f-4ee4-8066-586e3329a7da admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:02.021462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f93c83c6-fa03-4bcb-a8c4-d7f16e754f51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a28130cce5f641618d2bbb4023469951'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:02.047962 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 870/1725] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:02.049192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46132db6-836f-4ee4-8066-586e3329a7da admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:02.074393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46132db6-836f-4ee4-8066-586e3329a7da admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:02.086043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7461d-e628-453f-9ed4-aff28f473727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:02.087108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7461d-e628-453f-9ed4-aff28f473727 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:02.087569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7461d-e628-453f-9ed4-aff28f473727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:02.088058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7461d-e628-453f-9ed4-aff28f473727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:02.088878 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7461d-e628-453f-9ed4-aff28f473727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:02.090784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7461d-e628-453f-9ed4-aff28f473727 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:02.108965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 871/1726] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:02.113287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7461d-e628-453f-9ed4-aff28f473727 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:02.131532 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7461d-e628-453f-9ed4-aff28f473727 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:02.142910 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb2796-8306-4e9a-bd71-3802ff7448be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:02.144385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb2796-8306-4e9a-bd71-3802ff7448be admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:02.147304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb2796-8306-4e9a-bd71-3802ff7448be admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:02.147736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb2796-8306-4e9a-bd71-3802ff7448be admin admin] PATH_INFO: `/v3/projects/49e73e77cbf74e76a8c36241580fcbbb/users/63524f922338464ebb400b7c8d276021/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:02.188248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb2796-8306-4e9a-bd71-3802ff7448be admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e73e77cbf74e76a8c36241580fcbbb, user_id=63524f922338464ebb400b7c8d276021, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:02.204269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb2796-8306-4e9a-bd71-3802ff7448be admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:02.253384 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f93c83c6-fa03-4bcb-a8c4-d7f16e754f51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a28130cce5f641618d2bbb4023469951'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:02.254124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f93c83c6-fa03-4bcb-a8c4-d7f16e754f51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a28130cce5f641618d2bbb4023469951'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:02.278532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f93c83c6-fa03-4bcb-a8c4-d7f16e754f51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a28130cce5f641618d2bbb4023469951'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:02.287235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 872/1727] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:02 2022] PUT /identity/v3/projects/49e73e77cbf74e76a8c36241580fcbbb/users/63524f922338464ebb400b7c8d276021/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:02.299035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f677d-67f2-4ed3-b0d2-5e45b781d45f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:02.299830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f677d-67f2-4ed3-b0d2-5e45b781d45f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:02.300557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f677d-67f2-4ed3-b0d2-5e45b781d45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:02.314105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f93c83c6-fa03-4bcb-a8c4-d7f16e754f51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a28130cce5f641618d2bbb4023469951'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:02.329207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 856/1728] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:01 2022] DELETE /identity/v3/users/a28130cce5f641618d2bbb4023469951 => generated 0 bytes in 904 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:02.334153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada42de5-306d-488a-a614-defce8443d33 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:02.334624 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada42de5-306d-488a-a614-defce8443d33 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:02.335947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada42de5-306d-488a-a614-defce8443d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:02.351423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-824f677d-67f2-4ed3-b0d2-5e45b781d45f None None] MFA Rules not processed for user `63524f922338464ebb400b7c8d276021`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:02.352818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-824f677d-67f2-4ed3-b0d2-5e45b781d45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:02.383646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ada42de5-306d-488a-a614-defce8443d33 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:02.385057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ada42de5-306d-488a-a614-defce8443d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:02.473203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 873/1729] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:02.473787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41becd72-ec5d-4ccd-815a-27d17f5551b2 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:02.474848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41becd72-ec5d-4ccd-815a-27d17f5551b2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:02.475130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41becd72-ec5d-4ccd-815a-27d17f5551b2 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:02.479077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 874/1730] 10.222.0.57 () {54 vars in 912 bytes} [Tue Aug 9 18:49:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:49:02.483804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41becd72-ec5d-4ccd-815a-27d17f5551b2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:02.491250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-41becd72-ec5d-4ccd-815a-27d17f5551b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:02.495369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 857/1731] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:02.495369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24447a45-f597-40c5-8ec9-0cdfc3b65223 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:02.495369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24447a45-f597-40c5-8ec9-0cdfc3b65223 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:02.495369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24447a45-f597-40c5-8ec9-0cdfc3b65223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:02.531249 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41becd72-ec5d-4ccd-815a-27d17f5551b2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:02.539413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-24447a45-f597-40c5-8ec9-0cdfc3b65223 None None] MFA Rules not processed for user `88a7cdd610f74330a5da69c0de8472db`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:02.540764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-24447a45-f597-40c5-8ec9-0cdfc3b65223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:02.547847 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62fe45-b11b-4eb7-b509-e5d38fd49116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:02.548491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62fe45-b11b-4eb7-b509-e5d38fd49116 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:02.548762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62fe45-b11b-4eb7-b509-e5d38fd49116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:02.549083 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62fe45-b11b-4eb7-b509-e5d38fd49116 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:02.550634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62fe45-b11b-4eb7-b509-e5d38fd49116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:02.552687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62fe45-b11b-4eb7-b509-e5d38fd49116 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:02.618797 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 858/1732] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:49:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:02.620353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24447a45-f597-40c5-8ec9-0cdfc3b65223 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:02.622054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-24447a45-f597-40c5-8ec9-0cdfc3b65223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:02.634900 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24447a45-f597-40c5-8ec9-0cdfc3b65223 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:02.639474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f5dab-6570-4c44-bfbb-70159385d465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:02.640105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f5dab-6570-4c44-bfbb-70159385d465 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:02.640383 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f5dab-6570-4c44-bfbb-70159385d465 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:02.640656 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f5dab-6570-4c44-bfbb-70159385d465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:02.653371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f5dab-6570-4c44-bfbb-70159385d465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:02.655079 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f5dab-6570-4c44-bfbb-70159385d465 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:02.661989 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 875/1733] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:02 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:02.666374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62fe45-b11b-4eb7-b509-e5d38fd49116 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:02.691585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 859/1734] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:02.692520 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f5dab-6570-4c44-bfbb-70159385d465 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:02.727029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:02.734113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f5dab-6570-4c44-bfbb-70159385d465 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:02.786684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62fe45-b11b-4eb7-b509-e5d38fd49116 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:02.791467 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe16d4-db4c-42a5-8081-3619cee3dd68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:02.791915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe16d4-db4c-42a5-8081-3619cee3dd68 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:02.792196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe16d4-db4c-42a5-8081-3619cee3dd68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:02.797068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe16d4-db4c-42a5-8081-3619cee3dd68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:02.807607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe16d4-db4c-42a5-8081-3619cee3dd68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:02.811884 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe16d4-db4c-42a5-8081-3619cee3dd68 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:02.823295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 876/1735] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:02.839470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d190b82-8afa-4554-9c95-6f5bc4fe63d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:02.840156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d190b82-8afa-4554-9c95-6f5bc4fe63d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:02.840597 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d190b82-8afa-4554-9c95-6f5bc4fe63d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:02.840937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d190b82-8afa-4554-9c95-6f5bc4fe63d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:02.841611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d190b82-8afa-4554-9c95-6f5bc4fe63d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:02.844755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d190b82-8afa-4554-9c95-6f5bc4fe63d2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:02.954016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 860/1736] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:02 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:02.956077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe16d4-db4c-42a5-8081-3619cee3dd68 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:02.969361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe16d4-db4c-42a5-8081-3619cee3dd68 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:02.973008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:02.976359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdf95c-1459-4acd-8f42-eaa3b92a039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:02.977438 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdf95c-1459-4acd-8f42-eaa3b92a039d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:02.977772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdf95c-1459-4acd-8f42-eaa3b92a039d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:02.978045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdf95c-1459-4acd-8f42-eaa3b92a039d admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:02.978686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecdf95c-1459-4acd-8f42-eaa3b92a039d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:02.987123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecdf95c-1459-4acd-8f42-eaa3b92a039d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:02.995337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 877/1737] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:02.996495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d190b82-8afa-4554-9c95-6f5bc4fe63d2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:03.014947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d190b82-8afa-4554-9c95-6f5bc4fe63d2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:03.040372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362b75f-37f4-4a7e-8866-cdcabe3b1a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:03.041201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362b75f-37f4-4a7e-8866-cdcabe3b1a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:03.041632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362b75f-37f4-4a7e-8866-cdcabe3b1a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:03.042017 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362b75f-37f4-4a7e-8866-cdcabe3b1a2f admin admin] PATH_INFO: `/v3/projects/4ecd3b87899a43b3b8d35c428960a41c/users/3b36d5d3903f485a993fa0a934acf9e9/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:03.135739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362b75f-37f4-4a7e-8866-cdcabe3b1a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ecd3b87899a43b3b8d35c428960a41c, user_id=3b36d5d3903f485a993fa0a934acf9e9, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:03.135739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362b75f-37f4-4a7e-8866-cdcabe3b1a2f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:03.341015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 861/1738] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:02 2022] PUT /identity/v3/projects/4ecd3b87899a43b3b8d35c428960a41c/users/3b36d5d3903f485a993fa0a934acf9e9/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:03.346711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdf95c-1459-4acd-8f42-eaa3b92a039d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:03.363333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdf95c-1459-4acd-8f42-eaa3b92a039d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:03.531994 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219abb24-00ee-4797-b642-3b7932b90b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:03.536573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219abb24-00ee-4797-b642-3b7932b90b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:03.537191 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219abb24-00ee-4797-b642-3b7932b90b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:03.537886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219abb24-00ee-4797-b642-3b7932b90b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:03.540070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219abb24-00ee-4797-b642-3b7932b90b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:03.548939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219abb24-00ee-4797-b642-3b7932b90b4e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:03.588055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 878/1739] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:03.592380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362b75f-37f4-4a7e-8866-cdcabe3b1a2f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:03.612445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362b75f-37f4-4a7e-8866-cdcabe3b1a2f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:03.624484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31011b35-b564-41f6-aab4-ab09e8a07dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:03.625168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31011b35-b564-41f6-aab4-ab09e8a07dff admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:03.625452 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31011b35-b564-41f6-aab4-ab09e8a07dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:03.625842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31011b35-b564-41f6-aab4-ab09e8a07dff admin admin] PATH_INFO: `/v3/projects/4ecd3b87899a43b3b8d35c428960a41c/users/3b36d5d3903f485a993fa0a934acf9e9/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:03.631221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31011b35-b564-41f6-aab4-ab09e8a07dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ecd3b87899a43b3b8d35c428960a41c, user_id=3b36d5d3903f485a993fa0a934acf9e9, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:03.641798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31011b35-b564-41f6-aab4-ab09e8a07dff admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:03.670702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 862/1740] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:03 2022] PUT /identity/v3/projects/4ecd3b87899a43b3b8d35c428960a41c/users/3b36d5d3903f485a993fa0a934acf9e9/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:03.684059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eadb72-ceff-49df-8f61-6c616ee72dd0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:03.684393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eadb72-ceff-49df-8f61-6c616ee72dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:03.684673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eadb72-ceff-49df-8f61-6c616ee72dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:03.737610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-39eadb72-ceff-49df-8f61-6c616ee72dd0 None None] MFA Rules not processed for user `3b36d5d3903f485a993fa0a934acf9e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:03.739024 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-39eadb72-ceff-49df-8f61-6c616ee72dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:03.882925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 879/1741] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:03.896421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31011b35-b564-41f6-aab4-ab09e8a07dff admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:03.898639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-31011b35-b564-41f6-aab4-ab09e8a07dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:04.030493 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31011b35-b564-41f6-aab4-ab09e8a07dff admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:04.036439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e87a56-49ad-446c-bac4-468ce2f56396 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:04.040076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e87a56-49ad-446c-bac4-468ce2f56396 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:04.040076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e87a56-49ad-446c-bac4-468ce2f56396 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:04.040778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e87a56-49ad-446c-bac4-468ce2f56396 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:04.063931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e87a56-49ad-446c-bac4-468ce2f56396 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:04.067971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e87a56-49ad-446c-bac4-468ce2f56396 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:04.089995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 863/1742] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:04.113690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f3734-3bfc-4220-bcf5-0e70e4ad6310 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:04.114152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f3734-3bfc-4220-bcf5-0e70e4ad6310 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:04.114696 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f3734-3bfc-4220-bcf5-0e70e4ad6310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:04.162425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a20f3734-3bfc-4220-bcf5-0e70e4ad6310 None None] MFA Rules not processed for user `04da0f4276d6457c889ecb3006842e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:04.164175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a20f3734-3bfc-4220-bcf5-0e70e4ad6310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:04.229560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e87a56-49ad-446c-bac4-468ce2f56396 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:04.231429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-65e87a56-49ad-446c-bac4-468ce2f56396 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:04.321209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 880/1743] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:04.381098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e87a56-49ad-446c-bac4-468ce2f56396 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:04.386074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:04.387068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:04.390694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:04.390694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] PATH_INFO: `/v3/projects/98541d0a697a41bc8b1b865a3a5acaf2` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:04.390694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] RBAC: Authorizing `identity:delete_project(project_id=98541d0a697a41bc8b1b865a3a5acaf2)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:04.392167 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:04.507630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f3734-3bfc-4220-bcf5-0e70e4ad6310 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:04.515615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a20f3734-3bfc-4220-bcf5-0e70e4ad6310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:04.546319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f3734-3bfc-4220-bcf5-0e70e4ad6310 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:04.546438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98541d0a697a41bc8b1b865a3a5acaf2', 'request_id': 'req-41a193d9-7125-48a7-b7aa-34cbf3678ffb'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:04.573324 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98541d0a697a41bc8b1b865a3a5acaf2', 'request_id': 'req-41a193d9-7125-48a7-b7aa-34cbf3678ffb'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:04.580139 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98541d0a697a41bc8b1b865a3a5acaf2', 'request_id': 'req-41a193d9-7125-48a7-b7aa-34cbf3678ffb'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:04.641132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:04.646718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] The token cache is being invalidate because project 98541d0a697a41bc8b1b865a3a5acaf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:04.647140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:04.648109 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:04.649911 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 864/1744] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:04 2022] DELETE /identity/v3/projects/98541d0a697a41bc8b1b865a3a5acaf2 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:04.654145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:04.656190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:04.659213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489b846-cb58-4e66-b07a-48f7eb83ab49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:04.660809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489b846-cb58-4e66-b07a-48f7eb83ab49 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:04.661289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489b846-cb58-4e66-b07a-48f7eb83ab49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:04.665237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489b846-cb58-4e66-b07a-48f7eb83ab49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:04.665237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7489b846-cb58-4e66-b07a-48f7eb83ab49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:04.802297 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489b846-cb58-4e66-b07a-48f7eb83ab49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:04.804951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489b846-cb58-4e66-b07a-48f7eb83ab49 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:04.807853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a193d9-7125-48a7-b7aa-34cbf3678ffb admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:04.819304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a2730-71aa-4334-9288-b4778643e9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:04.821651 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a2730-71aa-4334-9288-b4778643e9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:04.822219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a2730-71aa-4334-9288-b4778643e9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:04.822323 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a2730-71aa-4334-9288-b4778643e9d9 admin admin] PATH_INFO: `/v3/users/3c7e13e7bdd24df38072faede964f8f9` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:04.825199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 881/1745] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:04.826275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a2730-71aa-4334-9288-b4778643e9d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c7e13e7bdd24df38072faede964f8f9)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:04.828337 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a2730-71aa-4334-9288-b4778643e9d9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:04.949665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:04.957072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055938f-8348-45d7-98e1-f7bc44714c5d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:04.961938 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055938f-8348-45d7-98e1-f7bc44714c5d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:04.961938 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055938f-8348-45d7-98e1-f7bc44714c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:05.003491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d055938f-8348-45d7-98e1-f7bc44714c5d None None] MFA Rules not processed for user `ef8af9fd9e844ca4be10caa880357a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:05.005246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d055938f-8348-45d7-98e1-f7bc44714c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:05.132290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 882/1746] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:05.136645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d95a2730-71aa-4334-9288-b4778643e9d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c7e13e7bdd24df38072faede964f8f9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:05.264938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d95a2730-71aa-4334-9288-b4778643e9d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c7e13e7bdd24df38072faede964f8f9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:05.265898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d95a2730-71aa-4334-9288-b4778643e9d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c7e13e7bdd24df38072faede964f8f9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:05.285026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d95a2730-71aa-4334-9288-b4778643e9d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c7e13e7bdd24df38072faede964f8f9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:05.362838 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d95a2730-71aa-4334-9288-b4778643e9d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c7e13e7bdd24df38072faede964f8f9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:05.381494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 865/1747] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:04 2022] DELETE /identity/v3/users/3c7e13e7bdd24df38072faede964f8f9 => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:06.393672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055938f-8348-45d7-98e1-f7bc44714c5d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:06.429628 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055938f-8348-45d7-98e1-f7bc44714c5d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:06.475139 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:06.543172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f58c80-1ce7-4790-baae-179b14bb562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:06.544017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f58c80-1ce7-4790-baae-179b14bb562d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:06.544750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f58c80-1ce7-4790-baae-179b14bb562d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:06.545171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f58c80-1ce7-4790-baae-179b14bb562d admin admin] PATH_INFO: `/v3/projects/90161cca676f4458937680f28d1f4a1c` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:06.547024 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f58c80-1ce7-4790-baae-179b14bb562d admin admin] RBAC: Authorizing `identity:delete_project(project_id=90161cca676f4458937680f28d1f4a1c)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:06.551966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f58c80-1ce7-4790-baae-179b14bb562d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:06.835403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c6f58c80-1ce7-4790-baae-179b14bb562d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90161cca676f4458937680f28d1f4a1c', 'request_id': 'req-c6f58c80-1ce7-4790-baae-179b14bb562d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:06.857762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c6f58c80-1ce7-4790-baae-179b14bb562d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90161cca676f4458937680f28d1f4a1c', 'request_id': 'req-c6f58c80-1ce7-4790-baae-179b14bb562d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:06.876534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c6f58c80-1ce7-4790-baae-179b14bb562d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90161cca676f4458937680f28d1f4a1c', 'request_id': 'req-c6f58c80-1ce7-4790-baae-179b14bb562d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:07.014791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c6f58c80-1ce7-4790-baae-179b14bb562d admin admin] The token cache is being invalidate because project 90161cca676f4458937680f28d1f4a1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:07.015386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c6f58c80-1ce7-4790-baae-179b14bb562d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:07.016730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c6f58c80-1ce7-4790-baae-179b14bb562d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:07.024632 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 883/1748] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:06 2022] DELETE /identity/v3/projects/90161cca676f4458937680f28d1f4a1c => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:07.027212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97258b-99c5-44f0-a487-0403cd82490a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:07.029236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97258b-99c5-44f0-a487-0403cd82490a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:07.029236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97258b-99c5-44f0-a487-0403cd82490a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:07.057269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ab97258b-99c5-44f0-a487-0403cd82490a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:07.059778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ab97258b-99c5-44f0-a487-0403cd82490a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:07.217815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597518c-b2e3-4ee6-b58a-f920617ac5a2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:07.218985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597518c-b2e3-4ee6-b58a-f920617ac5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:07.219530 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597518c-b2e3-4ee6-b58a-f920617ac5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:07.232834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 866/1749] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:07.288634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1597518c-b2e3-4ee6-b58a-f920617ac5a2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:07.290217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1597518c-b2e3-4ee6-b58a-f920617ac5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:07.307255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 884/1750] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:07.315742 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97258b-99c5-44f0-a487-0403cd82490a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:07.367272 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97258b-99c5-44f0-a487-0403cd82490a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:07.380018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb3564-82c4-4267-a335-2fa8e5512a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:07.380659 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb3564-82c4-4267-a335-2fa8e5512a35 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:07.380943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb3564-82c4-4267-a335-2fa8e5512a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:07.381216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb3564-82c4-4267-a335-2fa8e5512a35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:07.387377 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb3564-82c4-4267-a335-2fa8e5512a35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:07.387745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb3564-82c4-4267-a335-2fa8e5512a35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:07.397442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb3564-82c4-4267-a335-2fa8e5512a35 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:07.409799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672b310-16a2-4432-a8f8-2dfc4c6718bc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:07.410328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672b310-16a2-4432-a8f8-2dfc4c6718bc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:07.410405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672b310-16a2-4432-a8f8-2dfc4c6718bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:07.419181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 867/1751] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:07.425788 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb3564-82c4-4267-a335-2fa8e5512a35 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:07.441127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb3564-82c4-4267-a335-2fa8e5512a35 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:07.449955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb84af7-d6e7-4b98-a35c-1300e5bef62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:07.451159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb84af7-d6e7-4b98-a35c-1300e5bef62c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:07.452237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb84af7-d6e7-4b98-a35c-1300e5bef62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:07.453474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb84af7-d6e7-4b98-a35c-1300e5bef62c admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:07.454403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb84af7-d6e7-4b98-a35c-1300e5bef62c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:07.456672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb84af7-d6e7-4b98-a35c-1300e5bef62c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:07.475094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5672b310-16a2-4432-a8f8-2dfc4c6718bc None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:07.476365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5672b310-16a2-4432-a8f8-2dfc4c6718bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:07.576286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 868/1752] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:07 2022] POST /identity/v3/projects => generated 369 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:07.584823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb84af7-d6e7-4b98-a35c-1300e5bef62c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:07.607775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 885/1753] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:49:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:07.784569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672b310-16a2-4432-a8f8-2dfc4c6718bc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:07.787856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5672b310-16a2-4432-a8f8-2dfc4c6718bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:07.897509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:07.901464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb84af7-d6e7-4b98-a35c-1300e5bef62c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:07.962649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40b22f-e70e-4ef2-9df8-d8508abfd80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:07.964228 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40b22f-e70e-4ef2-9df8-d8508abfd80a admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:07.964553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40b22f-e70e-4ef2-9df8-d8508abfd80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:07.974778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40b22f-e70e-4ef2-9df8-d8508abfd80a admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:07.975537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40b22f-e70e-4ef2-9df8-d8508abfd80a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:07.982899 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40b22f-e70e-4ef2-9df8-d8508abfd80a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:07.992988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:08.099878 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672b310-16a2-4432-a8f8-2dfc4c6718bc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:08.185283 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692647a-a4b3-4091-b230-5caf2e2d7f4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:08.185948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692647a-a4b3-4091-b230-5caf2e2d7f4c service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:08.186234 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692647a-a4b3-4091-b230-5caf2e2d7f4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:08.186553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692647a-a4b3-4091-b230-5caf2e2d7f4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:08.189242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6692647a-a4b3-4091-b230-5caf2e2d7f4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:08.252987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 869/1754] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:07 2022] POST /identity/v3/users => generated 285 bytes in 672 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:08.255073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40b22f-e70e-4ef2-9df8-d8508abfd80a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:08.267130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0b40b22f-e70e-4ef2-9df8-d8508abfd80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:08.361392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692647a-a4b3-4091-b230-5caf2e2d7f4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:08.377959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692647a-a4b3-4091-b230-5caf2e2d7f4c service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:08.403794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 886/1755] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:08.408157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692647a-a4b3-4091-b230-5caf2e2d7f4c service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:08.440180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692647a-a4b3-4091-b230-5caf2e2d7f4c service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:08.451800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae223e-79f2-49ae-8bd7-154c1c0e7574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:08.453868 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae223e-79f2-49ae-8bd7-154c1c0e7574 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:08.454597 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae223e-79f2-49ae-8bd7-154c1c0e7574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:08.455887 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae223e-79f2-49ae-8bd7-154c1c0e7574 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:08.457299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae223e-79f2-49ae-8bd7-154c1c0e7574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:08.464644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae223e-79f2-49ae-8bd7-154c1c0e7574 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:08.496174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 887/1756] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:08.497735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae223e-79f2-49ae-8bd7-154c1c0e7574 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:08.519932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae223e-79f2-49ae-8bd7-154c1c0e7574 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:08.532105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386587a-bfe0-45b1-bbfe-c2076b69224b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:08.532892 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386587a-bfe0-45b1-bbfe-c2076b69224b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:08.533280 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386587a-bfe0-45b1-bbfe-c2076b69224b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:08.534203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386587a-bfe0-45b1-bbfe-c2076b69224b admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:08.535353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386587a-bfe0-45b1-bbfe-c2076b69224b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:08.542688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386587a-bfe0-45b1-bbfe-c2076b69224b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:08.575397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 888/1757] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:08.579540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386587a-bfe0-45b1-bbfe-c2076b69224b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:08.597366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40b22f-e70e-4ef2-9df8-d8508abfd80a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:08.599678 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386587a-bfe0-45b1-bbfe-c2076b69224b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:08.604490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dda70-c5ac-44fd-97fe-a7a0ed0d502f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:08.605323 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dda70-c5ac-44fd-97fe-a7a0ed0d502f service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:08.605808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dda70-c5ac-44fd-97fe-a7a0ed0d502f service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:08.605937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b57a0-caa3-4526-bcab-6f3197a9bc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:08.606155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dda70-c5ac-44fd-97fe-a7a0ed0d502f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:08.606650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b57a0-caa3-4526-bcab-6f3197a9bc53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:08.606927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b57a0-caa3-4526-bcab-6f3197a9bc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:08.607199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b57a0-caa3-4526-bcab-6f3197a9bc53 admin admin] PATH_INFO: `/v3/projects/b50d2c73ae544a24aa518e4529cbc7cd/users/5376a1cab3444d55bcd8d1531eabce3a/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:08.642662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b57a0-caa3-4526-bcab-6f3197a9bc53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b50d2c73ae544a24aa518e4529cbc7cd, user_id=5376a1cab3444d55bcd8d1531eabce3a, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:08.647378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b57a0-caa3-4526-bcab-6f3197a9bc53 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:08.692375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259dda70-c5ac-44fd-97fe-a7a0ed0d502f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:08.693957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259dda70-c5ac-44fd-97fe-a7a0ed0d502f service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:08.719176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 870/1758] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:49:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:49:08.779961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 889/1759] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:08 2022] PUT /identity/v3/projects/b50d2c73ae544a24aa518e4529cbc7cd/users/5376a1cab3444d55bcd8d1531eabce3a/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:08.793548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662aa6-8006-4f91-b9f7-059a19b1e595 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:08.794000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662aa6-8006-4f91-b9f7-059a19b1e595 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:08.794445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662aa6-8006-4f91-b9f7-059a19b1e595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:08.853168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1f662aa6-8006-4f91-b9f7-059a19b1e595 None None] MFA Rules not processed for user `5376a1cab3444d55bcd8d1531eabce3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:08.855776 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1f662aa6-8006-4f91-b9f7-059a19b1e595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:09.130535 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 871/1760] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:09.135327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b57a0-caa3-4526-bcab-6f3197a9bc53 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:09.488090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b57a0-caa3-4526-bcab-6f3197a9bc53 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:09.521385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28820453-7fad-4b30-8f28-9683357b62b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:09.522242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28820453-7fad-4b30-8f28-9683357b62b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:09.523552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28820453-7fad-4b30-8f28-9683357b62b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:09.524026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28820453-7fad-4b30-8f28-9683357b62b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:09.532556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28820453-7fad-4b30-8f28-9683357b62b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:09.536363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28820453-7fad-4b30-8f28-9683357b62b7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:09.667375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256a26-47a5-4a9a-9427-14997fe51a02 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:09.667948 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256a26-47a5-4a9a-9427-14997fe51a02 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:09.668049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256a26-47a5-4a9a-9427-14997fe51a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:09.711841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 890/1761] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:09 2022] POST /identity/v3/projects => generated 367 bytes in 577 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:09.716427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28820453-7fad-4b30-8f28-9683357b62b7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:09.772747 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-17256a26-47a5-4a9a-9427-14997fe51a02 None None] MFA Rules not processed for user `88a7cdd610f74330a5da69c0de8472db`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:09.774402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-17256a26-47a5-4a9a-9427-14997fe51a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:10.084242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 872/1762] 10.222.0.45 () {58 vars in 1015 bytes} [Tue Aug 9 18:49:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:10.102852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256a26-47a5-4a9a-9427-14997fe51a02 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:10.104639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-17256a26-47a5-4a9a-9427-14997fe51a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:10.108474 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28820453-7fad-4b30-8f28-9683357b62b7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:10.127590 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26962602-a95e-44ac-b393-726830839516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:10.127801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256a26-47a5-4a9a-9427-14997fe51a02 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:10.128722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26962602-a95e-44ac-b393-726830839516 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:10.129209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26962602-a95e-44ac-b393-726830839516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:10.137447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26962602-a95e-44ac-b393-726830839516 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:10.139082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26962602-a95e-44ac-b393-726830839516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:10.139202 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4394e52-3b13-4091-92a4-7660a4459e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:10.139865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394e52-3b13-4091-92a4-7660a4459e8c service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:10.140209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394e52-3b13-4091-92a4-7660a4459e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:10.140549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394e52-3b13-4091-92a4-7660a4459e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:10.142278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26962602-a95e-44ac-b393-726830839516 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:10.166989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4394e52-3b13-4091-92a4-7660a4459e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:10.168590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4394e52-3b13-4091-92a4-7660a4459e8c service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:10.196138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 873/1763] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:10.242821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 891/1764] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:09 2022] POST /identity/v3/users => generated 290 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:10.245161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4394e52-3b13-4091-92a4-7660a4459e8c service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:10.302310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4394e52-3b13-4091-92a4-7660a4459e8c service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:10.318324 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a34b56-9b40-489f-a6a6-b4a12f96a76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:10.319325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a34b56-9b40-489f-a6a6-b4a12f96a76a admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:10.319740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a34b56-9b40-489f-a6a6-b4a12f96a76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:10.320265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a34b56-9b40-489f-a6a6-b4a12f96a76a admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:10.321063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a34b56-9b40-489f-a6a6-b4a12f96a76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:10.324610 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a34b56-9b40-489f-a6a6-b4a12f96a76a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:10.352876 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 874/1765] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:10.356928 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26962602-a95e-44ac-b393-726830839516 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:10.375763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26962602-a95e-44ac-b393-726830839516 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:10.389850 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dc518-1436-4504-8245-255693b5ba81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:10.390708 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dc518-1436-4504-8245-255693b5ba81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:10.391150 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dc518-1436-4504-8245-255693b5ba81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:10.391564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dc518-1436-4504-8245-255693b5ba81 admin admin] PATH_INFO: `/v3/projects/5ba9a512cd2f4e1dbd1f897ceb83b009/users/0fcec4590a4b4c33bafeb9f979a4c906/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:10.429496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a34b56-9b40-489f-a6a6-b4a12f96a76a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:10.433850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-22a34b56-9b40-489f-a6a6-b4a12f96a76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:10.456170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dc518-1436-4504-8245-255693b5ba81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba9a512cd2f4e1dbd1f897ceb83b009, user_id=0fcec4590a4b4c33bafeb9f979a4c906, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:10.460082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dc518-1436-4504-8245-255693b5ba81 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:10.468690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a34b56-9b40-489f-a6a6-b4a12f96a76a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:10.477105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360af724-246e-4e81-87e3-056df8a4ac51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:10.482482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360af724-246e-4e81-87e3-056df8a4ac51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:10.483088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360af724-246e-4e81-87e3-056df8a4ac51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:10.483518 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360af724-246e-4e81-87e3-056df8a4ac51 admin admin] PATH_INFO: `/v3/users/0f93cab77f2b4bfd839c4e08ddfa1da7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:10.485148 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360af724-246e-4e81-87e3-056df8a4ac51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f93cab77f2b4bfd839c4e08ddfa1da7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:10.564712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360af724-246e-4e81-87e3-056df8a4ac51 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:10.928581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 892/1766] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:10 2022] PUT /identity/v3/projects/5ba9a512cd2f4e1dbd1f897ceb83b009/users/0fcec4590a4b4c33bafeb9f979a4c906/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:10.945021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dc518-1436-4504-8245-255693b5ba81 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:11.079705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dc518-1436-4504-8245-255693b5ba81 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:11.086422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed145e-b387-457f-90d6-155bef4e3d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:11.087546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed145e-b387-457f-90d6-155bef4e3d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:11.088148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed145e-b387-457f-90d6-155bef4e3d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:11.088878 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed145e-b387-457f-90d6-155bef4e3d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:11.090251 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed145e-b387-457f-90d6-155bef4e3d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:11.092670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed145e-b387-457f-90d6-155bef4e3d92 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:11.113275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 893/1767] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:11.119221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed145e-b387-457f-90d6-155bef4e3d92 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:11.133571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed145e-b387-457f-90d6-155bef4e3d92 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:11.144039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704c90e-ad96-428b-b1ea-779780673b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:11.146995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704c90e-ad96-428b-b1ea-779780673b81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:11.148657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704c90e-ad96-428b-b1ea-779780673b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:11.149382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704c90e-ad96-428b-b1ea-779780673b81 admin admin] PATH_INFO: `/v3/projects/5ba9a512cd2f4e1dbd1f897ceb83b009/users/0fcec4590a4b4c33bafeb9f979a4c906/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:11.158262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704c90e-ad96-428b-b1ea-779780673b81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba9a512cd2f4e1dbd1f897ceb83b009, user_id=0fcec4590a4b4c33bafeb9f979a4c906, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:11.164523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704c90e-ad96-428b-b1ea-779780673b81 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:11.184285 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-360af724-246e-4e81-87e3-056df8a4ac51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f93cab77f2b4bfd839c4e08ddfa1da7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:11.197643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 894/1768] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:11 2022] PUT /identity/v3/projects/5ba9a512cd2f4e1dbd1f897ceb83b009/users/0fcec4590a4b4c33bafeb9f979a4c906/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:11.209194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de42fc3-9cdd-45df-b13b-c534bc595f65 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:11.209787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de42fc3-9cdd-45df-b13b-c534bc595f65 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:11.210330 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de42fc3-9cdd-45df-b13b-c534bc595f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:11.286668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0de42fc3-9cdd-45df-b13b-c534bc595f65 None None] MFA Rules not processed for user `0fcec4590a4b4c33bafeb9f979a4c906`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:11.286668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0de42fc3-9cdd-45df-b13b-c534bc595f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:11.305611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-360af724-246e-4e81-87e3-056df8a4ac51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f93cab77f2b4bfd839c4e08ddfa1da7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:11.307107 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-360af724-246e-4e81-87e3-056df8a4ac51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f93cab77f2b4bfd839c4e08ddfa1da7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:11.329407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-360af724-246e-4e81-87e3-056df8a4ac51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f93cab77f2b4bfd839c4e08ddfa1da7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:11.355335 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-360af724-246e-4e81-87e3-056df8a4ac51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f93cab77f2b4bfd839c4e08ddfa1da7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:11.367712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 875/1769] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:10 2022] DELETE /identity/v3/users/0f93cab77f2b4bfd839c4e08ddfa1da7 => generated 0 bytes in 939 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:11.373972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f543d-072e-460b-aa2a-5c6c8c832faf None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:11.374519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f543d-072e-460b-aa2a-5c6c8c832faf None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:11.374987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f543d-072e-460b-aa2a-5c6c8c832faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:11.403547 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-bb6f543d-072e-460b-aa2a-5c6c8c832faf None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:11.405189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bb6f543d-072e-460b-aa2a-5c6c8c832faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:11.435096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 895/1770] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:11.466272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de42fc3-9cdd-45df-b13b-c534bc595f65 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:11.467797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0de42fc3-9cdd-45df-b13b-c534bc595f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:11.522164 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 876/1771] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:11.526981 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:11.563217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de42fc3-9cdd-45df-b13b-c534bc595f65 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:11.663411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f543d-072e-460b-aa2a-5c6c8c832faf None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:11.666269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356a8cc-da45-49a1-99a3-b5e4c1e81e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:11.667307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356a8cc-da45-49a1-99a3-b5e4c1e81e2d service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:11.667307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356a8cc-da45-49a1-99a3-b5e4c1e81e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:11.667452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356a8cc-da45-49a1-99a3-b5e4c1e81e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:11.667894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bb6f543d-072e-460b-aa2a-5c6c8c832faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:11.704915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:11.778161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356a8cc-da45-49a1-99a3-b5e4c1e81e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:11.783151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356a8cc-da45-49a1-99a3-b5e4c1e81e2d service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:11.818032 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 896/1772] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:11.862994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f543d-072e-460b-aa2a-5c6c8c832faf None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:11.871846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b7c0f-4bc4-4717-8364-8a757225f948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:11.872739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b7c0f-4bc4-4717-8364-8a757225f948 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:11.873772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b7c0f-4bc4-4717-8364-8a757225f948 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:11.874372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b7c0f-4bc4-4717-8364-8a757225f948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:11.881551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b347800-5ed3-4479-801d-84a8e6308334 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:11.881913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b347800-5ed3-4479-801d-84a8e6308334 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:11.882194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b347800-5ed3-4479-801d-84a8e6308334 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:11.883862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 897/1773] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:49:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:49:11.895940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c60b8-afb2-42ae-be05-44d06ee13bed None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:11.897577 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c60b8-afb2-42ae-be05-44d06ee13bed None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:11.897799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c60b8-afb2-42ae-be05-44d06ee13bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:11.898802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b7c0f-4bc4-4717-8364-8a757225f948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:11.903870 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b7c0f-4bc4-4717-8364-8a757225f948 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:11.927977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 877/1774] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:11.964570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b7c0f-4bc4-4717-8364-8a757225f948 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:11.966589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-666b7c0f-4bc4-4717-8364-8a757225f948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:11.991517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-428c60b8-afb2-42ae-be05-44d06ee13bed None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:11.993126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-428c60b8-afb2-42ae-be05-44d06ee13bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:11.997360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b7c0f-4bc4-4717-8364-8a757225f948 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:12.008875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec765c-8959-44d9-a8b2-c4c1babcfc1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:12.009848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec765c-8959-44d9-a8b2-c4c1babcfc1c service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:12.010824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec765c-8959-44d9-a8b2-c4c1babcfc1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:12.010824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec765c-8959-44d9-a8b2-c4c1babcfc1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:12.015978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-34ec765c-8959-44d9-a8b2-c4c1babcfc1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:12.112219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 898/1775] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:49:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:12.133861 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c60b8-afb2-42ae-be05-44d06ee13bed None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:12.135952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-428c60b8-afb2-42ae-be05-44d06ee13bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:12.152140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec765c-8959-44d9-a8b2-c4c1babcfc1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:12.153867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec765c-8959-44d9-a8b2-c4c1babcfc1c service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:12.157146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c60b8-afb2-42ae-be05-44d06ee13bed None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:12.165233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998f032-3915-4b23-8f78-8bfd64ed81c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:12.166546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998f032-3915-4b23-8f78-8bfd64ed81c0 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:12.167182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 878/1776] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:12.167499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998f032-3915-4b23-8f78-8bfd64ed81c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:12.167922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998f032-3915-4b23-8f78-8bfd64ed81c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:12.178860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998f032-3915-4b23-8f78-8bfd64ed81c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:12.180578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998f032-3915-4b23-8f78-8bfd64ed81c0 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:12.200766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 899/1777] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:12.841127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:12.929245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec765c-8959-44d9-a8b2-c4c1babcfc1c service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:12.930729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-34ec765c-8959-44d9-a8b2-c4c1babcfc1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:12.944412 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec765c-8959-44d9-a8b2-c4c1babcfc1c service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:12.949115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:12.949852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:12.950085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:12.950359 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] PATH_INFO: `/v3/projects/bfec9cc41c7346acb8b4405c8603b07c` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:12.951488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfec9cc41c7346acb8b4405c8603b07c)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:12.953418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:13.024142 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:13.096736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfec9cc41c7346acb8b4405c8603b07c', 'request_id': 'req-7cd98233-c37e-438a-93ff-65a45b544f39'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:13.114112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfec9cc41c7346acb8b4405c8603b07c', 'request_id': 'req-7cd98233-c37e-438a-93ff-65a45b544f39'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:13.129496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfec9cc41c7346acb8b4405c8603b07c', 'request_id': 'req-7cd98233-c37e-438a-93ff-65a45b544f39'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:13.224219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] The token cache is being invalidate because project bfec9cc41c7346acb8b4405c8603b07c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:13.228241 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:13.228241 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:13.228241 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 879/1778] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:12 2022] DELETE /identity/v3/projects/bfec9cc41c7346acb8b4405c8603b07c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:13.231246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998f032-3915-4b23-8f78-8bfd64ed81c0 service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:13.244056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998f032-3915-4b23-8f78-8bfd64ed81c0 service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:13.363359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a8170-14eb-4708-8f44-0a6e944f44a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:13.364977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a8170-14eb-4708-8f44-0a6e944f44a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:13.365396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a8170-14eb-4708-8f44-0a6e944f44a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:13.365790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a8170-14eb-4708-8f44-0a6e944f44a6 admin admin] PATH_INFO: `/v3/users/a1e9d7912646464cb9a860dd692c829a` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:13.367170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a8170-14eb-4708-8f44-0a6e944f44a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1e9d7912646464cb9a860dd692c829a)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:13.369257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a8170-14eb-4708-8f44-0a6e944f44a6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:13.588799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f49a8170-14eb-4708-8f44-0a6e944f44a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1e9d7912646464cb9a860dd692c829a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:13.663512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f49a8170-14eb-4708-8f44-0a6e944f44a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1e9d7912646464cb9a860dd692c829a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:13.664134 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f49a8170-14eb-4708-8f44-0a6e944f44a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1e9d7912646464cb9a860dd692c829a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:13.684396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f49a8170-14eb-4708-8f44-0a6e944f44a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1e9d7912646464cb9a860dd692c829a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:13.700652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f49a8170-14eb-4708-8f44-0a6e944f44a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1e9d7912646464cb9a860dd692c829a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:13.707992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 900/1779] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:13 2022] DELETE /identity/v3/users/a1e9d7912646464cb9a860dd692c829a => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:14.229790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:14.343053 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:14.345448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:14.364300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd98233-c37e-438a-93ff-65a45b544f39 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:14.463470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01613ba5-3da9-46c9-ab9c-bc1352e3e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:14.464106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01613ba5-3da9-46c9-ab9c-bc1352e3e906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:14.464386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01613ba5-3da9-46c9-ab9c-bc1352e3e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:14.464659 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01613ba5-3da9-46c9-ab9c-bc1352e3e906 admin admin] PATH_INFO: `/v3/projects/fead135916ef452da4ab72d4ca01fff9` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:14.465886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01613ba5-3da9-46c9-ab9c-bc1352e3e906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fead135916ef452da4ab72d4ca01fff9)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:14.467822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01613ba5-3da9-46c9-ab9c-bc1352e3e906 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:14.598606 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-01613ba5-3da9-46c9-ab9c-bc1352e3e906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fead135916ef452da4ab72d4ca01fff9', 'request_id': 'req-01613ba5-3da9-46c9-ab9c-bc1352e3e906'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:14.607574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-01613ba5-3da9-46c9-ab9c-bc1352e3e906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fead135916ef452da4ab72d4ca01fff9', 'request_id': 'req-01613ba5-3da9-46c9-ab9c-bc1352e3e906'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:14.615580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-01613ba5-3da9-46c9-ab9c-bc1352e3e906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fead135916ef452da4ab72d4ca01fff9', 'request_id': 'req-01613ba5-3da9-46c9-ab9c-bc1352e3e906'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:14.637915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:14.715773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-01613ba5-3da9-46c9-ab9c-bc1352e3e906 admin admin] The token cache is being invalidate because project fead135916ef452da4ab72d4ca01fff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:14.718705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-01613ba5-3da9-46c9-ab9c-bc1352e3e906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:14.718705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-01613ba5-3da9-46c9-ab9c-bc1352e3e906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:14.719361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 880/1780] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:14 2022] DELETE /identity/v3/projects/fead135916ef452da4ab72d4ca01fff9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:14.725282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63273a3-2c9f-4b11-82e5-835a9d4a17be None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:14.725638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63273a3-2c9f-4b11-82e5-835a9d4a17be None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:14.725921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63273a3-2c9f-4b11-82e5-835a9d4a17be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:14.750477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b63273a3-2c9f-4b11-82e5-835a9d4a17be None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:14.752531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b63273a3-2c9f-4b11-82e5-835a9d4a17be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:14.809097 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:14.849026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 901/1781] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:14.858662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549206b-b2a8-4be2-a68d-1c4947c32c71 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:14.858951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549206b-b2a8-4be2-a68d-1c4947c32c71 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:14.859260 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549206b-b2a8-4be2-a68d-1c4947c32c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:14.882033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3549206b-b2a8-4be2-a68d-1c4947c32c71 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:14.883283 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3549206b-b2a8-4be2-a68d-1c4947c32c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:14.891861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 881/1782] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:14.896261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63273a3-2c9f-4b11-82e5-835a9d4a17be None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:14.912778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63273a3-2c9f-4b11-82e5-835a9d4a17be None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:14.918900 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2c058-a134-45c8-97f5-3f4a76c05efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:14.919393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2c058-a134-45c8-97f5-3f4a76c05efa admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:14.919552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2c058-a134-45c8-97f5-3f4a76c05efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:14.919812 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2c058-a134-45c8-97f5-3f4a76c05efa admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:14.920483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2c058-a134-45c8-97f5-3f4a76c05efa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:14.920790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2c058-a134-45c8-97f5-3f4a76c05efa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:14.922475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2c058-a134-45c8-97f5-3f4a76c05efa admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:14.941987 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 902/1783] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:14.943431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549206b-b2a8-4be2-a68d-1c4947c32c71 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:14.960394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549206b-b2a8-4be2-a68d-1c4947c32c71 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:14.971478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f8186-8429-4628-8309-95d3fef077d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:14.972520 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f8186-8429-4628-8309-95d3fef077d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:14.972897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f8186-8429-4628-8309-95d3fef077d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:14.973275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f8186-8429-4628-8309-95d3fef077d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:14.989092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f8186-8429-4628-8309-95d3fef077d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:14.994837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f8186-8429-4628-8309-95d3fef077d6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:15.112127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 882/1784] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:14 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:15.116237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2c058-a134-45c8-97f5-3f4a76c05efa admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:15.139888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2c058-a134-45c8-97f5-3f4a76c05efa admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:15.263725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a4352-aba8-4e44-947f-fa2caf580066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:15.264382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a4352-aba8-4e44-947f-fa2caf580066 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:15.264566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a4352-aba8-4e44-947f-fa2caf580066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:15.264843 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a4352-aba8-4e44-947f-fa2caf580066 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:15.265499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a4352-aba8-4e44-947f-fa2caf580066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:15.268024 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a4352-aba8-4e44-947f-fa2caf580066 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:15.358030 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 903/1785] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:15 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:15.359576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f8186-8429-4628-8309-95d3fef077d6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:15.374200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f8186-8429-4628-8309-95d3fef077d6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:15.389838 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23500b48-77a9-4469-ab4c-9fab94e300f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:15.410721 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23500b48-77a9-4469-ab4c-9fab94e300f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:15.411415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23500b48-77a9-4469-ab4c-9fab94e300f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:15.411705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23500b48-77a9-4469-ab4c-9fab94e300f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:15.412399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23500b48-77a9-4469-ab4c-9fab94e300f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:15.419357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23500b48-77a9-4469-ab4c-9fab94e300f7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:15.454595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 883/1786] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:15.460639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a4352-aba8-4e44-947f-fa2caf580066 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:15.481765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a4352-aba8-4e44-947f-fa2caf580066 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:15.490182 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ed2fb-b6c6-4a33-873b-af6e5f731c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:15.490824 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ed2fb-b6c6-4a33-873b-af6e5f731c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:15.491113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ed2fb-b6c6-4a33-873b-af6e5f731c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:15.491390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ed2fb-b6c6-4a33-873b-af6e5f731c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:15.492065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ed2fb-b6c6-4a33-873b-af6e5f731c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:15.493794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ed2fb-b6c6-4a33-873b-af6e5f731c25 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:15.508272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 904/1787] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:49:15.513380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23500b48-77a9-4469-ab4c-9fab94e300f7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:15.534940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23500b48-77a9-4469-ab4c-9fab94e300f7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:15.544854 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b9ba6-5dcb-4c75-98f0-43dc02cf0503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:15.545888 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b9ba6-5dcb-4c75-98f0-43dc02cf0503 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:15.546441 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b9ba6-5dcb-4c75-98f0-43dc02cf0503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:15.546738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b9ba6-5dcb-4c75-98f0-43dc02cf0503 admin admin] PATH_INFO: `/v3/projects/3907f1c5094b4cb68464311e544f11c1/users/04c2f8262a1b4f2383063cc9497d157b/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:15.567067 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b9ba6-5dcb-4c75-98f0-43dc02cf0503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3907f1c5094b4cb68464311e544f11c1, user_id=04c2f8262a1b4f2383063cc9497d157b, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:15.570319 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b9ba6-5dcb-4c75-98f0-43dc02cf0503 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:15.664719 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 884/1788] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:15 2022] PUT /identity/v3/projects/3907f1c5094b4cb68464311e544f11c1/users/04c2f8262a1b4f2383063cc9497d157b/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:15.675788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2dc91-9b91-4d91-bc4a-92b651dfbdf3 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:15.676075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2dc91-9b91-4d91-bc4a-92b651dfbdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:15.676345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2dc91-9b91-4d91-bc4a-92b651dfbdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:15.715365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-72f2dc91-9b91-4d91-bc4a-92b651dfbdf3 None None] MFA Rules not processed for user `04c2f8262a1b4f2383063cc9497d157b`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:15.717180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-72f2dc91-9b91-4d91-bc4a-92b651dfbdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:15.750117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:15.813489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 905/1789] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:15.815097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b9ba6-5dcb-4c75-98f0-43dc02cf0503 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:15.886552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b9ba6-5dcb-4c75-98f0-43dc02cf0503 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:15.892365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59de817-6ad9-4ca3-8f52-e66685aad5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:15.893752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59de817-6ad9-4ca3-8f52-e66685aad5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:15.894124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59de817-6ad9-4ca3-8f52-e66685aad5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:15.894461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59de817-6ad9-4ca3-8f52-e66685aad5f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:15.895178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59de817-6ad9-4ca3-8f52-e66685aad5f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:15.897260 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59de817-6ad9-4ca3-8f52-e66685aad5f3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:16.045245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 885/1790] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:15 2022] POST /identity/v3/projects => generated 387 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:16.049351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2dc91-9b91-4d91-bc4a-92b651dfbdf3 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:16.073074 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2dc91-9b91-4d91-bc4a-92b651dfbdf3 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:16.160615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc70843-3491-41eb-a453-056d4796c147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:16.161326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc70843-3491-41eb-a453-056d4796c147 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:16.161720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc70843-3491-41eb-a453-056d4796c147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:16.162052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc70843-3491-41eb-a453-056d4796c147 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:16.163183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc70843-3491-41eb-a453-056d4796c147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:16.165256 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc70843-3491-41eb-a453-056d4796c147 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:16.268949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 906/1791] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:16 2022] POST /identity/v3/users => generated 294 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:16.273385 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59de817-6ad9-4ca3-8f52-e66685aad5f3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:16.285202 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59de817-6ad9-4ca3-8f52-e66685aad5f3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:16.290234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715dc6c-0428-4bdd-ade1-1bfcd0074022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:16.290975 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715dc6c-0428-4bdd-ade1-1bfcd0074022 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:16.291104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715dc6c-0428-4bdd-ade1-1bfcd0074022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:16.291364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715dc6c-0428-4bdd-ade1-1bfcd0074022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:16.292002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715dc6c-0428-4bdd-ade1-1bfcd0074022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:16.293621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715dc6c-0428-4bdd-ade1-1bfcd0074022 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:16.305430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 886/1792] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:16.309955 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc70843-3491-41eb-a453-056d4796c147 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:16.322664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc70843-3491-41eb-a453-056d4796c147 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:16.328237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49c354-bca6-4881-84f9-ff063cb7c221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:16.329071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49c354-bca6-4881-84f9-ff063cb7c221 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:16.329357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49c354-bca6-4881-84f9-ff063cb7c221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:16.329761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49c354-bca6-4881-84f9-ff063cb7c221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:16.331161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49c354-bca6-4881-84f9-ff063cb7c221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:16.332768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49c354-bca6-4881-84f9-ff063cb7c221 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:16.353944 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715dc6c-0428-4bdd-ade1-1bfcd0074022 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:16.354346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 907/1793] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:16.362553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:16.366778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715dc6c-0428-4bdd-ade1-1bfcd0074022 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:16.371757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b33b81-69d7-40a2-9a87-c8b162824033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:16.372554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b33b81-69d7-40a2-9a87-c8b162824033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:16.372979 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b33b81-69d7-40a2-9a87-c8b162824033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:16.373503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b33b81-69d7-40a2-9a87-c8b162824033 admin admin] PATH_INFO: `/v3/projects/50f4d55608f140e3b7782a9d09431513/users/deeee1670f774ed8afa93e77c8a48c3c/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:16.390892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b33b81-69d7-40a2-9a87-c8b162824033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50f4d55608f140e3b7782a9d09431513, user_id=deeee1670f774ed8afa93e77c8a48c3c, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:16.394275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b33b81-69d7-40a2-9a87-c8b162824033 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:16.520997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:16.524484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 887/1794] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:16 2022] PUT /identity/v3/projects/50f4d55608f140e3b7782a9d09431513/users/deeee1670f774ed8afa93e77c8a48c3c/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:16.538332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65ecdd-ef94-41d3-8e37-5b895e27b989 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:16.538648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65ecdd-ef94-41d3-8e37-5b895e27b989 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:16.538978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65ecdd-ef94-41d3-8e37-5b895e27b989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:16.615713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cd65ecdd-ef94-41d3-8e37-5b895e27b989 None None] MFA Rules not processed for user `deeee1670f774ed8afa93e77c8a48c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:16.617180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cd65ecdd-ef94-41d3-8e37-5b895e27b989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:16.733575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 908/1795] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:16.738287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b33b81-69d7-40a2-9a87-c8b162824033 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:16.858968 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b33b81-69d7-40a2-9a87-c8b162824033 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:16.865076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca875de-f607-47a3-bf71-30b1f6a9acc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:16.865823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca875de-f607-47a3-bf71-30b1f6a9acc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:16.866114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca875de-f607-47a3-bf71-30b1f6a9acc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:16.866392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca875de-f607-47a3-bf71-30b1f6a9acc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:16.867417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca875de-f607-47a3-bf71-30b1f6a9acc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:16.869352 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca875de-f607-47a3-bf71-30b1f6a9acc9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:17.066453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 888/1796] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:16 2022] POST /identity/v3/projects => generated 387 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:17.072654 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65ecdd-ef94-41d3-8e37-5b895e27b989 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:17.084948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65ecdd-ef94-41d3-8e37-5b895e27b989 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:17.163059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a77c2-e276-4a49-89bb-59b8a208ec93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:17.165739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a77c2-e276-4a49-89bb-59b8a208ec93 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:17.165739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a77c2-e276-4a49-89bb-59b8a208ec93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:17.165739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a77c2-e276-4a49-89bb-59b8a208ec93 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:17.165739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a77c2-e276-4a49-89bb-59b8a208ec93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:17.167505 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a77c2-e276-4a49-89bb-59b8a208ec93 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:17.309956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 909/1797] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:17 2022] POST /identity/v3/users => generated 300 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:17.313301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca875de-f607-47a3-bf71-30b1f6a9acc9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:17.325020 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca875de-f607-47a3-bf71-30b1f6a9acc9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:17.330359 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ec87b-1c0b-4070-9c88-04b586bed2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:17.331322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ec87b-1c0b-4070-9c88-04b586bed2f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:17.332491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ec87b-1c0b-4070-9c88-04b586bed2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:17.332491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ec87b-1c0b-4070-9c88-04b586bed2f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:17.332491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ec87b-1c0b-4070-9c88-04b586bed2f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:17.334369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ec87b-1c0b-4070-9c88-04b586bed2f8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:17.346534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 889/1798] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:17.353157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a77c2-e276-4a49-89bb-59b8a208ec93 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:17.368479 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a77c2-e276-4a49-89bb-59b8a208ec93 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:17.383346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0be34-26e1-46f7-96bc-891aa7e398cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:17.384214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0be34-26e1-46f7-96bc-891aa7e398cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:17.385722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0be34-26e1-46f7-96bc-891aa7e398cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:17.385722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0be34-26e1-46f7-96bc-891aa7e398cb admin admin] PATH_INFO: `/v3/projects/308df6c13ef64ce8992f61f31abea8f7/users/df806c112adb4a06944b45a66b937ddd/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:17.416596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0be34-26e1-46f7-96bc-891aa7e398cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=308df6c13ef64ce8992f61f31abea8f7, user_id=df806c112adb4a06944b45a66b937ddd, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:17.420293 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0be34-26e1-46f7-96bc-891aa7e398cb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:17.476845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 910/1799] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:17 2022] PUT /identity/v3/projects/308df6c13ef64ce8992f61f31abea8f7/users/df806c112adb4a06944b45a66b937ddd/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:17.480604 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ec87b-1c0b-4070-9c88-04b586bed2f8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:17.492545 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ec87b-1c0b-4070-9c88-04b586bed2f8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:17.562503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db8c13-d3c8-4cb2-94fe-665855bacd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:17.563332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db8c13-d3c8-4cb2-94fe-665855bacd29 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:17.563688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db8c13-d3c8-4cb2-94fe-665855bacd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:17.564035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db8c13-d3c8-4cb2-94fe-665855bacd29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:17.565712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db8c13-d3c8-4cb2-94fe-665855bacd29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:17.567441 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db8c13-d3c8-4cb2-94fe-665855bacd29 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:17.581228 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 890/1800] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:17.582416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0be34-26e1-46f7-96bc-891aa7e398cb admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:17.593946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0be34-26e1-46f7-96bc-891aa7e398cb admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:17.598860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762d00e-587c-48f6-9184-816a256e10ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:17.599933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762d00e-587c-48f6-9184-816a256e10ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:17.600223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762d00e-587c-48f6-9184-816a256e10ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:17.600503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762d00e-587c-48f6-9184-816a256e10ea admin admin] PATH_INFO: `/v3/projects/308df6c13ef64ce8992f61f31abea8f7/users/df806c112adb4a06944b45a66b937ddd/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:17.602756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762d00e-587c-48f6-9184-816a256e10ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=308df6c13ef64ce8992f61f31abea8f7, user_id=df806c112adb4a06944b45a66b937ddd, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:17.606130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762d00e-587c-48f6-9184-816a256e10ea admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:17.620009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 911/1801] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:17 2022] PUT /identity/v3/projects/308df6c13ef64ce8992f61f31abea8f7/users/df806c112adb4a06944b45a66b937ddd/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:17.630995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ac77-f6bf-430b-a785-a2424bb9a798 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:17.631276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ac77-f6bf-430b-a785-a2424bb9a798 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:17.631549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ac77-f6bf-430b-a785-a2424bb9a798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:17.661729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f6f4ac77-f6bf-430b-a785-a2424bb9a798 None None] MFA Rules not processed for user `df806c112adb4a06944b45a66b937ddd`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:17.663358 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f6f4ac77-f6bf-430b-a785-a2424bb9a798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:17.757214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:17.777168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 891/1802] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:17.789820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762d00e-587c-48f6-9184-816a256e10ea admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:17.792088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b762d00e-587c-48f6-9184-816a256e10ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:17.866337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762d00e-587c-48f6-9184-816a256e10ea admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:17.886849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bcb9c-980e-462c-9301-7af57e9e1bfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:17.887524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bcb9c-980e-462c-9301-7af57e9e1bfd service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:17.887867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bcb9c-980e-462c-9301-7af57e9e1bfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:17.888209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bcb9c-980e-462c-9301-7af57e9e1bfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:17.922972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bcb9c-980e-462c-9301-7af57e9e1bfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:17.926322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bcb9c-980e-462c-9301-7af57e9e1bfd service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:17.926751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:17.942676 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4ac77-f6bf-430b-a785-a2424bb9a798 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:17.944329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f6f4ac77-f6bf-430b-a785-a2424bb9a798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:17.946083 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 912/1803] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:18.084338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4ac77-f6bf-430b-a785-a2424bb9a798 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:18.092927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:18.093650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:18.093931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:18.099412 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] PATH_INFO: `/v3/users/5376a1cab3444d55bcd8d1531eabce3a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:18.099412 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5376a1cab3444d55bcd8d1531eabce3a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:18.099412 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:18.387229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5376a1cab3444d55bcd8d1531eabce3a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:18.459062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5376a1cab3444d55bcd8d1531eabce3a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:18.459910 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5376a1cab3444d55bcd8d1531eabce3a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:18.470992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5376a1cab3444d55bcd8d1531eabce3a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:18.487773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5376a1cab3444d55bcd8d1531eabce3a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:18.510133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 892/1804] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:17 2022] DELETE /identity/v3/users/5376a1cab3444d55bcd8d1531eabce3a => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:18.512101 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014b57d-33ce-4528-92dc-2b850c2d1447 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:18.512598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014b57d-33ce-4528-92dc-2b850c2d1447 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:18.513573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014b57d-33ce-4528-92dc-2b850c2d1447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:18.545158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a014b57d-33ce-4528-92dc-2b850c2d1447 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:18.546498 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a014b57d-33ce-4528-92dc-2b850c2d1447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:18.759278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 913/1805] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:18.771828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:18.773414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:18.897333 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ff9b4-2e36-4f8d-a398-883ccb5d56e0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:18.902253 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473aeef-5a21-49fa-b66d-9b5c93b087b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:18.903459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473aeef-5a21-49fa-b66d-9b5c93b087b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:18.903876 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473aeef-5a21-49fa-b66d-9b5c93b087b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:18.904026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473aeef-5a21-49fa-b66d-9b5c93b087b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:18.924863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473aeef-5a21-49fa-b66d-9b5c93b087b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:18.926761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473aeef-5a21-49fa-b66d-9b5c93b087b1 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:18.939436 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 893/1806] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:20.372673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014b57d-33ce-4528-92dc-2b850c2d1447 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:20.376985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a014b57d-33ce-4528-92dc-2b850c2d1447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:20.429778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:20.473266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014b57d-33ce-4528-92dc-2b850c2d1447 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:20.487221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e200929-799e-4e48-8324-f4e563e2794c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:20.487221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e200929-799e-4e48-8324-f4e563e2794c service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:20.487221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e200929-799e-4e48-8324-f4e563e2794c service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:20.487221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e200929-799e-4e48-8324-f4e563e2794c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:20.491980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8e200929-799e-4e48-8324-f4e563e2794c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:20.526287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:20.592392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e200929-799e-4e48-8324-f4e563e2794c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:20.593828 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e200929-799e-4e48-8324-f4e563e2794c service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:20.609720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 914/1807] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:20.713277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473aeef-5a21-49fa-b66d-9b5c93b087b1 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:20.715429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a473aeef-5a21-49fa-b66d-9b5c93b087b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:20.741317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473aeef-5a21-49fa-b66d-9b5c93b087b1 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:20.792796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:20.794779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:20.796480 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:20.796759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a admin admin] PATH_INFO: `/v3/projects/b50d2c73ae544a24aa518e4529cbc7cd` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:20.800467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b50d2c73ae544a24aa518e4529cbc7cd)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:20.805201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:20.935218 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b50d2c73ae544a24aa518e4529cbc7cd', 'request_id': 'req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:20.946369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b50d2c73ae544a24aa518e4529cbc7cd', 'request_id': 'req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:20.953990 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b50d2c73ae544a24aa518e4529cbc7cd', 'request_id': 'req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:21.080206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a admin admin] The token cache is being invalidate because project b50d2c73ae544a24aa518e4529cbc7cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:21.080617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:21.081415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c0361ff3-b146-41ec-878d-47a9c1c5fb8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:21.083200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 894/1808] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:20 2022] DELETE /identity/v3/projects/b50d2c73ae544a24aa518e4529cbc7cd => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:21.088209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e200929-799e-4e48-8324-f4e563e2794c service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:21.102848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e200929-799e-4e48-8324-f4e563e2794c service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:21.171029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014c0fb-adb9-442a-a40e-b392da5e2d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:21.171711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014c0fb-adb9-442a-a40e-b392da5e2d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:21.171991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014c0fb-adb9-442a-a40e-b392da5e2d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:21.172286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014c0fb-adb9-442a-a40e-b392da5e2d8e admin admin] PATH_INFO: `/v3/users/0fcec4590a4b4c33bafeb9f979a4c906` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:21.173426 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014c0fb-adb9-442a-a40e-b392da5e2d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fcec4590a4b4c33bafeb9f979a4c906)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:21.176187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014c0fb-adb9-442a-a40e-b392da5e2d8e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:21.408755 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b014c0fb-adb9-442a-a40e-b392da5e2d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fcec4590a4b4c33bafeb9f979a4c906'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:21.469056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b014c0fb-adb9-442a-a40e-b392da5e2d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fcec4590a4b4c33bafeb9f979a4c906'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:21.469849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b014c0fb-adb9-442a-a40e-b392da5e2d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fcec4590a4b4c33bafeb9f979a4c906'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:21.478469 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b014c0fb-adb9-442a-a40e-b392da5e2d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fcec4590a4b4c33bafeb9f979a4c906'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:21.495010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b014c0fb-adb9-442a-a40e-b392da5e2d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fcec4590a4b4c33bafeb9f979a4c906'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:21.514166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 915/1809] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:21 2022] DELETE /identity/v3/users/0fcec4590a4b4c33bafeb9f979a4c906 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:21.869486 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b91c1a-ef92-4c04-8092-a148b9ac9b79 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:21.870059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b91c1a-ef92-4c04-8092-a148b9ac9b79 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:21.870258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b91c1a-ef92-4c04-8092-a148b9ac9b79 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:21.871733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 895/1810] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:49:21.895594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242aeb2-2dcd-495e-b4e6-7fb99bda972e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:21.896293 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242aeb2-2dcd-495e-b4e6-7fb99bda972e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:21.896946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242aeb2-2dcd-495e-b4e6-7fb99bda972e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:21.918764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8242aeb2-2dcd-495e-b4e6-7fb99bda972e None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:21.920140 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8242aeb2-2dcd-495e-b4e6-7fb99bda972e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:21.973479 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:21.987005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b91c1a-ef92-4c04-8092-a148b9ac9b79 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:21.990002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-86b91c1a-ef92-4c04-8092-a148b9ac9b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:22.023649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 916/1811] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:49:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:22.048217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242aeb2-2dcd-495e-b4e6-7fb99bda972e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:22.051379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8242aeb2-2dcd-495e-b4e6-7fb99bda972e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:22.067720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242aeb2-2dcd-495e-b4e6-7fb99bda972e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:22.073886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f512913-73a2-43f6-a6eb-1b249b0c3262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:22.074658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f512913-73a2-43f6-a6eb-1b249b0c3262 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:22.075011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f512913-73a2-43f6-a6eb-1b249b0c3262 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:22.075364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f512913-73a2-43f6-a6eb-1b249b0c3262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:22.086981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f512913-73a2-43f6-a6eb-1b249b0c3262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:22.088685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f512913-73a2-43f6-a6eb-1b249b0c3262 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:22.102367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 917/1812] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:22.112715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b91c1a-ef92-4c04-8092-a148b9ac9b79 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:22.128934 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7582f4-ef74-4942-ac8a-8548e7798c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:22.129836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7582f4-ef74-4942-ac8a-8548e7798c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:22.130727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7582f4-ef74-4942-ac8a-8548e7798c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:22.131328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7582f4-ef74-4942-ac8a-8548e7798c35 admin admin] PATH_INFO: `/v3/projects/5ba9a512cd2f4e1dbd1f897ceb83b009` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:22.132736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7582f4-ef74-4942-ac8a-8548e7798c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba9a512cd2f4e1dbd1f897ceb83b009)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:22.135544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7582f4-ef74-4942-ac8a-8548e7798c35 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:22.290114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4a7582f4-ef74-4942-ac8a-8548e7798c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba9a512cd2f4e1dbd1f897ceb83b009', 'request_id': 'req-4a7582f4-ef74-4942-ac8a-8548e7798c35'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:22.302106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4a7582f4-ef74-4942-ac8a-8548e7798c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba9a512cd2f4e1dbd1f897ceb83b009', 'request_id': 'req-4a7582f4-ef74-4942-ac8a-8548e7798c35'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:22.319292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4a7582f4-ef74-4942-ac8a-8548e7798c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba9a512cd2f4e1dbd1f897ceb83b009', 'request_id': 'req-4a7582f4-ef74-4942-ac8a-8548e7798c35'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:22.445914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4a7582f4-ef74-4942-ac8a-8548e7798c35 admin admin] The token cache is being invalidate because project 5ba9a512cd2f4e1dbd1f897ceb83b009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:22.446207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4a7582f4-ef74-4942-ac8a-8548e7798c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:22.446854 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4a7582f4-ef74-4942-ac8a-8548e7798c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:22.448395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 896/1813] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:21 2022] DELETE /identity/v3/projects/5ba9a512cd2f4e1dbd1f897ceb83b009 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:22.453367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fda6b-0238-45e7-be06-902c248219a0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:22.453854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fda6b-0238-45e7-be06-902c248219a0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:22.453999 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fda6b-0238-45e7-be06-902c248219a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:22.475188 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-832fda6b-0238-45e7-be06-902c248219a0 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:22.476385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-832fda6b-0238-45e7-be06-902c248219a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:22.526708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:22.535231 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 918/1814] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:22.543658 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef274afe-91f3-48bd-9d9d-80882274de82 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:22.543957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef274afe-91f3-48bd-9d9d-80882274de82 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:22.544213 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef274afe-91f3-48bd-9d9d-80882274de82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:22.569101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ef274afe-91f3-48bd-9d9d-80882274de82 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:22.572836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ef274afe-91f3-48bd-9d9d-80882274de82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:22.578518 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 897/1815] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:22.583467 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fda6b-0238-45e7-be06-902c248219a0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:22.594604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fda6b-0238-45e7-be06-902c248219a0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:22.601002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3e351-97f5-4a4d-a062-0b08442d17b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:22.607311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3e351-97f5-4a4d-a062-0b08442d17b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:22.607311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3e351-97f5-4a4d-a062-0b08442d17b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:22.607311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3e351-97f5-4a4d-a062-0b08442d17b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:22.607311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3e351-97f5-4a4d-a062-0b08442d17b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:22.607311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3e351-97f5-4a4d-a062-0b08442d17b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:22.607311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3e351-97f5-4a4d-a062-0b08442d17b6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:22.627869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 919/1816] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:22.631271 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef274afe-91f3-48bd-9d9d-80882274de82 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:22.652983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef274afe-91f3-48bd-9d9d-80882274de82 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:22.652983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36260b-f3a8-47a6-9916-b5cba1203ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:22.653462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36260b-f3a8-47a6-9916-b5cba1203ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:22.653552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36260b-f3a8-47a6-9916-b5cba1203ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:22.653861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36260b-f3a8-47a6-9916-b5cba1203ee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:22.654524 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36260b-f3a8-47a6-9916-b5cba1203ee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:22.656552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36260b-f3a8-47a6-9916-b5cba1203ee7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:22.808498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 898/1817] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:22 2022] POST /identity/v3/projects => generated 389 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:22.812991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3e351-97f5-4a4d-a062-0b08442d17b6 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:22.825147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3e351-97f5-4a4d-a062-0b08442d17b6 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:22.893476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e736243-bedb-45ec-b525-d1bf3272339f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:22.894389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e736243-bedb-45ec-b525-d1bf3272339f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:22.894389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e736243-bedb-45ec-b525-d1bf3272339f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:22.894677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e736243-bedb-45ec-b525-d1bf3272339f admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:22.895593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e736243-bedb-45ec-b525-d1bf3272339f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:22.897660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e736243-bedb-45ec-b525-d1bf3272339f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:23.043354 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 920/1818] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:22 2022] POST /identity/v3/users => generated 295 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:23.048190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36260b-f3a8-47a6-9916-b5cba1203ee7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:23.074624 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36260b-f3a8-47a6-9916-b5cba1203ee7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:23.088670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67be438-655b-4552-8ccd-1799d9936a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:23.089313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67be438-655b-4552-8ccd-1799d9936a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:23.089626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67be438-655b-4552-8ccd-1799d9936a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:23.093409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67be438-655b-4552-8ccd-1799d9936a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:23.093409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67be438-655b-4552-8ccd-1799d9936a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:23.096987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67be438-655b-4552-8ccd-1799d9936a81 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:23.166400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 899/1819] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:23.166618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e736243-bedb-45ec-b525-d1bf3272339f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:23.197208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e736243-bedb-45ec-b525-d1bf3272339f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:23.203642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158f096-52d7-4061-8dc0-4c334f9b24ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:23.204287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158f096-52d7-4061-8dc0-4c334f9b24ad admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:23.204703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158f096-52d7-4061-8dc0-4c334f9b24ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:23.205491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158f096-52d7-4061-8dc0-4c334f9b24ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:23.206211 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158f096-52d7-4061-8dc0-4c334f9b24ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:23.207869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158f096-52d7-4061-8dc0-4c334f9b24ad admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:23.223022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 921/1820] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:23.224476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67be438-655b-4552-8ccd-1799d9936a81 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:23.239712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67be438-655b-4552-8ccd-1799d9936a81 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:23.245238 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989ad2c-f046-4182-9cef-1d2f1f87b6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:23.245967 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989ad2c-f046-4182-9cef-1d2f1f87b6e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:23.246251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989ad2c-f046-4182-9cef-1d2f1f87b6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:23.246530 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989ad2c-f046-4182-9cef-1d2f1f87b6e9 admin admin] PATH_INFO: `/v3/projects/27a2280d0ebc47aebb1ceda18cc5f2d2/users/8eca787bf76641b38d795271f3a18c9a/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:23.270136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989ad2c-f046-4182-9cef-1d2f1f87b6e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a2280d0ebc47aebb1ceda18cc5f2d2, user_id=8eca787bf76641b38d795271f3a18c9a, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:23.278279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989ad2c-f046-4182-9cef-1d2f1f87b6e9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:23.637056 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 900/1821] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:23 2022] PUT /identity/v3/projects/27a2280d0ebc47aebb1ceda18cc5f2d2/users/8eca787bf76641b38d795271f3a18c9a/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:23.651785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef201b-32cb-4e82-8b20-7ac5b0fcbace None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:23.652891 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef201b-32cb-4e82-8b20-7ac5b0fcbace None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:23.653391 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef201b-32cb-4e82-8b20-7ac5b0fcbace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:23.725294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2aef201b-32cb-4e82-8b20-7ac5b0fcbace None None] MFA Rules not processed for user `8eca787bf76641b38d795271f3a18c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:23.726793 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2aef201b-32cb-4e82-8b20-7ac5b0fcbace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:23.833868 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 922/1822] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:23.837997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989ad2c-f046-4182-9cef-1d2f1f87b6e9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:23.903987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989ad2c-f046-4182-9cef-1d2f1f87b6e9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:23.909547 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda3162-de16-4402-894c-46ef95df75a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:23.910173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda3162-de16-4402-894c-46ef95df75a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:23.910727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda3162-de16-4402-894c-46ef95df75a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:23.910727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda3162-de16-4402-894c-46ef95df75a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:23.911352 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda3162-de16-4402-894c-46ef95df75a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:23.913323 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda3162-de16-4402-894c-46ef95df75a3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:24.001893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 901/1823] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:23 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:24.003428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef201b-32cb-4e82-8b20-7ac5b0fcbace None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:24.015069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef201b-32cb-4e82-8b20-7ac5b0fcbace None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:24.072651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9197675-9a36-42e8-bb9a-b44cb8a36605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:24.073454 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9197675-9a36-42e8-bb9a-b44cb8a36605 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:24.073764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9197675-9a36-42e8-bb9a-b44cb8a36605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:24.074034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9197675-9a36-42e8-bb9a-b44cb8a36605 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:24.074648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9197675-9a36-42e8-bb9a-b44cb8a36605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:24.076652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9197675-9a36-42e8-bb9a-b44cb8a36605 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:24.155579 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 923/1824] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:24 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:24.160182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda3162-de16-4402-894c-46ef95df75a3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:24.171471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda3162-de16-4402-894c-46ef95df75a3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:24.178689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be959d-88f8-4b13-973a-27fb6d3cd1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:24.178689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be959d-88f8-4b13-973a-27fb6d3cd1e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:24.178689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be959d-88f8-4b13-973a-27fb6d3cd1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:24.178689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be959d-88f8-4b13-973a-27fb6d3cd1e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:24.179158 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be959d-88f8-4b13-973a-27fb6d3cd1e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:24.179862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be959d-88f8-4b13-973a-27fb6d3cd1e7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:24.191890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 902/1825] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:24.201870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9197675-9a36-42e8-bb9a-b44cb8a36605 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:24.208862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9197675-9a36-42e8-bb9a-b44cb8a36605 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:24.217487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c3046-b11d-4874-9a0d-77515e890c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:24.229317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c3046-b11d-4874-9a0d-77515e890c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:24.229317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c3046-b11d-4874-9a0d-77515e890c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:24.229317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c3046-b11d-4874-9a0d-77515e890c5d admin admin] PATH_INFO: `/v3/projects/10a0a3cb403a46f08a74a5ea71558be1/users/9d64c6c53cde4199b44ed9a9180e60da/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:24.234630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c3046-b11d-4874-9a0d-77515e890c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a0a3cb403a46f08a74a5ea71558be1, user_id=9d64c6c53cde4199b44ed9a9180e60da, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:24.237335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c3046-b11d-4874-9a0d-77515e890c5d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:24.320420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 924/1826] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:24 2022] PUT /identity/v3/projects/10a0a3cb403a46f08a74a5ea71558be1/users/9d64c6c53cde4199b44ed9a9180e60da/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:24.324512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be959d-88f8-4b13-973a-27fb6d3cd1e7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:24.334820 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be959d-88f8-4b13-973a-27fb6d3cd1e7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:24.380011 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:24.401000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cf5bf-4f3d-41c9-9638-b76f300178ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:24.401653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf5bf-4f3d-41c9-9638-b76f300178ab admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:24.404780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf5bf-4f3d-41c9-9638-b76f300178ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:24.404780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf5bf-4f3d-41c9-9638-b76f300178ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:24.406439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cf5bf-4f3d-41c9-9638-b76f300178ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:24.410302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cf5bf-4f3d-41c9-9638-b76f300178ab admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:24.428228 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 903/1827] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:24.429427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c3046-b11d-4874-9a0d-77515e890c5d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:24.440596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c3046-b11d-4874-9a0d-77515e890c5d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:24.445514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47602c1-9897-47bf-982c-872b07e289e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:24.446200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47602c1-9897-47bf-982c-872b07e289e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:24.446475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47602c1-9897-47bf-982c-872b07e289e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:24.446746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47602c1-9897-47bf-982c-872b07e289e1 admin admin] PATH_INFO: `/v3/projects/10a0a3cb403a46f08a74a5ea71558be1/users/9d64c6c53cde4199b44ed9a9180e60da/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:24.448673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47602c1-9897-47bf-982c-872b07e289e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a0a3cb403a46f08a74a5ea71558be1, user_id=9d64c6c53cde4199b44ed9a9180e60da, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:24.451671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47602c1-9897-47bf-982c-872b07e289e1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:24.464102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 925/1828] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:24 2022] PUT /identity/v3/projects/10a0a3cb403a46f08a74a5ea71558be1/users/9d64c6c53cde4199b44ed9a9180e60da/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:24.476634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fe020-c75c-4a9c-bf5d-3dcaffd5b3f3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:24.476928 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fe020-c75c-4a9c-bf5d-3dcaffd5b3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:24.477205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fe020-c75c-4a9c-bf5d-3dcaffd5b3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:24.510108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-133fe020-c75c-4a9c-bf5d-3dcaffd5b3f3 None None] MFA Rules not processed for user `9d64c6c53cde4199b44ed9a9180e60da`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:24.511604 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-133fe020-c75c-4a9c-bf5d-3dcaffd5b3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:24.531697 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:24.589316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 904/1829] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:24.600856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47602c1-9897-47bf-982c-872b07e289e1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:24.602763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d47602c1-9897-47bf-982c-872b07e289e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:24.671049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47602c1-9897-47bf-982c-872b07e289e1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:24.676070 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5eb147-ef9e-458f-9e5d-304657a93798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:24.676702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5eb147-ef9e-458f-9e5d-304657a93798 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:24.676984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5eb147-ef9e-458f-9e5d-304657a93798 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:24.677255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5eb147-ef9e-458f-9e5d-304657a93798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:24.697940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5eb147-ef9e-458f-9e5d-304657a93798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:24.699402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5eb147-ef9e-458f-9e5d-304657a93798 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:24.714065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 926/1830] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:24.913236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fe020-c75c-4a9c-bf5d-3dcaffd5b3f3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:24.983765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fe020-c75c-4a9c-bf5d-3dcaffd5b3f3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:24.990822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:24.991954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:24.992435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:24.992926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] PATH_INFO: `/v3/users/8eca787bf76641b38d795271f3a18c9a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:24.996198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eca787bf76641b38d795271f3a18c9a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:24.998432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:25.303149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eca787bf76641b38d795271f3a18c9a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:25.389457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eca787bf76641b38d795271f3a18c9a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:25.392835 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eca787bf76641b38d795271f3a18c9a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:25.401123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eca787bf76641b38d795271f3a18c9a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:25.418376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eca787bf76641b38d795271f3a18c9a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:25.426163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 905/1831] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:24 2022] DELETE /identity/v3/users/8eca787bf76641b38d795271f3a18c9a => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:25.432113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f5303-7d37-48a3-9340-bd8c01c84973 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:25.437112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f5303-7d37-48a3-9340-bd8c01c84973 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:25.437112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f5303-7d37-48a3-9340-bd8c01c84973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:25.461912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4d4f5303-7d37-48a3-9340-bd8c01c84973 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:25.463139 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4d4f5303-7d37-48a3-9340-bd8c01c84973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:25.545208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 927/1832] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:25.564500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:25.568536 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:25.652669 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2148c-47c6-4741-9817-5a85135eafd9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:25.659454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44971b24-ff42-47f1-9252-fb284df9b60b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:25.660435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44971b24-ff42-47f1-9252-fb284df9b60b service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:25.660876 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44971b24-ff42-47f1-9252-fb284df9b60b service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:25.661326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44971b24-ff42-47f1-9252-fb284df9b60b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:25.672445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44971b24-ff42-47f1-9252-fb284df9b60b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:25.674261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44971b24-ff42-47f1-9252-fb284df9b60b service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:25.685495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 906/1833] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:25.749378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:25.983786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:26.271724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f5303-7d37-48a3-9340-bd8c01c84973 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:26.273557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4d4f5303-7d37-48a3-9340-bd8c01c84973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:26.286077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f5303-7d37-48a3-9340-bd8c01c84973 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:26.291390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933d987-ec57-4f03-a545-2d6c9b95d643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:26.292023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933d987-ec57-4f03-a545-2d6c9b95d643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:26.292306 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933d987-ec57-4f03-a545-2d6c9b95d643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:26.292585 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933d987-ec57-4f03-a545-2d6c9b95d643 admin admin] PATH_INFO: `/v3/users/63524f922338464ebb400b7c8d276021` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:26.293835 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933d987-ec57-4f03-a545-2d6c9b95d643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63524f922338464ebb400b7c8d276021)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:26.295828 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933d987-ec57-4f03-a545-2d6c9b95d643 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:26.455470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44971b24-ff42-47f1-9252-fb284df9b60b service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:26.457258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-44971b24-ff42-47f1-9252-fb284df9b60b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:26.472216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44971b24-ff42-47f1-9252-fb284df9b60b service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:26.477236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:26.477969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:26.478262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:26.478540 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] PATH_INFO: `/v3/projects/27a2280d0ebc47aebb1ceda18cc5f2d2` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:26.479696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] RBAC: Authorizing `identity:delete_project(project_id=27a2280d0ebc47aebb1ceda18cc5f2d2)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:26.481889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:26.605268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9933d987-ec57-4f03-a545-2d6c9b95d643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63524f922338464ebb400b7c8d276021'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:26.622731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27a2280d0ebc47aebb1ceda18cc5f2d2', 'request_id': 'req-96c2c617-13e7-41d4-a92f-a08dfdca076f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:26.630080 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27a2280d0ebc47aebb1ceda18cc5f2d2', 'request_id': 'req-96c2c617-13e7-41d4-a92f-a08dfdca076f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:26.639722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27a2280d0ebc47aebb1ceda18cc5f2d2', 'request_id': 'req-96c2c617-13e7-41d4-a92f-a08dfdca076f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:26.721062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9933d987-ec57-4f03-a545-2d6c9b95d643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63524f922338464ebb400b7c8d276021'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:26.721816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9933d987-ec57-4f03-a545-2d6c9b95d643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63524f922338464ebb400b7c8d276021'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:26.730564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9933d987-ec57-4f03-a545-2d6c9b95d643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63524f922338464ebb400b7c8d276021'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:26.746337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9933d987-ec57-4f03-a545-2d6c9b95d643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63524f922338464ebb400b7c8d276021'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:26.754354 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 928/1834] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:26 2022] DELETE /identity/v3/users/63524f922338464ebb400b7c8d276021 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:26.759605 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2cedc-f4b2-413d-918e-44df4dd62a70 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:26.759899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2cedc-f4b2-413d-918e-44df4dd62a70 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:26.760180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2cedc-f4b2-413d-918e-44df4dd62a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:26.789934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-01d2cedc-f4b2-413d-918e-44df4dd62a70 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:26.789934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-01d2cedc-f4b2-413d-918e-44df4dd62a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:26.812549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] The token cache is being invalidate because project 27a2280d0ebc47aebb1ceda18cc5f2d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:26.819009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:26.819009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:26.819009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 907/1835] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:26 2022] DELETE /identity/v3/projects/27a2280d0ebc47aebb1ceda18cc5f2d2 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:26.821388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:26.824399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:26.888116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 929/1836] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:26.899031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2cedc-f4b2-413d-918e-44df4dd62a70 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:26.903044 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2c617-13e7-41d4-a92f-a08dfdca076f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:26.903169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-01d2cedc-f4b2-413d-918e-44df4dd62a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:26.905054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:26.905814 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:26.906100 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:26.906377 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] PATH_INFO: `/v3/users/9d64c6c53cde4199b44ed9a9180e60da` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:26.908219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d64c6c53cde4199b44ed9a9180e60da)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:26.910380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:26.965469 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2cedc-f4b2-413d-918e-44df4dd62a70 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:26.971214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea145ff-0368-4d8a-981f-5bd04fc78a70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:26.972001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea145ff-0368-4d8a-981f-5bd04fc78a70 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:26.972351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea145ff-0368-4d8a-981f-5bd04fc78a70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:26.972720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea145ff-0368-4d8a-981f-5bd04fc78a70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:26.995550 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea145ff-0368-4d8a-981f-5bd04fc78a70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:26.996432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea145ff-0368-4d8a-981f-5bd04fc78a70 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:27.009072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 930/1837] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:27.035305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:27.149937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d64c6c53cde4199b44ed9a9180e60da'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:27.255934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea145ff-0368-4d8a-981f-5bd04fc78a70 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:27.257500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7ea145ff-0368-4d8a-981f-5bd04fc78a70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:27.270789 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d64c6c53cde4199b44ed9a9180e60da'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:27.271700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d64c6c53cde4199b44ed9a9180e60da'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:27.281093 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d64c6c53cde4199b44ed9a9180e60da'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:27.310870 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d64c6c53cde4199b44ed9a9180e60da'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:27.322476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 908/1838] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:26 2022] DELETE /identity/v3/users/9d64c6c53cde4199b44ed9a9180e60da => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:27.336915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea145ff-0368-4d8a-981f-5bd04fc78a70 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:27.392935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2669b34-ab24-4246-9877-a6d82e26a0b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:27.393905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2669b34-ab24-4246-9877-a6d82e26a0b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:27.394391 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2669b34-ab24-4246-9877-a6d82e26a0b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:27.394813 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2669b34-ab24-4246-9877-a6d82e26a0b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:27.398785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c2669b34-ab24-4246-9877-a6d82e26a0b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:27.489967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2669b34-ab24-4246-9877-a6d82e26a0b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:27.491393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2669b34-ab24-4246-9877-a6d82e26a0b6 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:27.502803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 931/1839] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:27.571326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:27.573345 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:27.659216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed7a3d-c694-48f2-8527-b5cb532df63b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:27.664055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:27.664696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:27.664969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:27.666147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] PATH_INFO: `/v3/projects/49e73e77cbf74e76a8c36241580fcbbb` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:27.667968 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49e73e77cbf74e76a8c36241580fcbbb)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:27.670454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:27.732732 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:27.778497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49e73e77cbf74e76a8c36241580fcbbb', 'request_id': 'req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:27.786347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49e73e77cbf74e76a8c36241580fcbbb', 'request_id': 'req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:27.793798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49e73e77cbf74e76a8c36241580fcbbb', 'request_id': 'req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:27.870903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2669b34-ab24-4246-9877-a6d82e26a0b6 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:27.872564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c2669b34-ab24-4246-9877-a6d82e26a0b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:27.887041 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2669b34-ab24-4246-9877-a6d82e26a0b6 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:27.902695 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] The token cache is being invalidate because project 49e73e77cbf74e76a8c36241580fcbbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:27.903017 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:27.903722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:27.908808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 909/1840] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:27 2022] DELETE /identity/v3/projects/49e73e77cbf74e76a8c36241580fcbbb => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:27.913074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:27.914463 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:27.963460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123e05b-74d7-4d00-803c-9795e7a5ada1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:27.964133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123e05b-74d7-4d00-803c-9795e7a5ada1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:27.964418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123e05b-74d7-4d00-803c-9795e7a5ada1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:27.964692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123e05b-74d7-4d00-803c-9795e7a5ada1 admin admin] PATH_INFO: `/v3/projects/10a0a3cb403a46f08a74a5ea71558be1` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:27.966537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123e05b-74d7-4d00-803c-9795e7a5ada1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10a0a3cb403a46f08a74a5ea71558be1)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:27.968549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123e05b-74d7-4d00-803c-9795e7a5ada1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:28.000399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d75e8-ed62-4e02-82e1-ce1efcbc9107 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:28.007511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:28.012895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:28.012895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:28.012895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] PATH_INFO: `/v3/users/3b36d5d3903f485a993fa0a934acf9e9` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:28.012895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b36d5d3903f485a993fa0a934acf9e9)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:28.012895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:28.097151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6123e05b-74d7-4d00-803c-9795e7a5ada1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10a0a3cb403a46f08a74a5ea71558be1', 'request_id': 'req-6123e05b-74d7-4d00-803c-9795e7a5ada1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:28.102692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6123e05b-74d7-4d00-803c-9795e7a5ada1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10a0a3cb403a46f08a74a5ea71558be1', 'request_id': 'req-6123e05b-74d7-4d00-803c-9795e7a5ada1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:28.109327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6123e05b-74d7-4d00-803c-9795e7a5ada1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10a0a3cb403a46f08a74a5ea71558be1', 'request_id': 'req-6123e05b-74d7-4d00-803c-9795e7a5ada1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:28.268246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6123e05b-74d7-4d00-803c-9795e7a5ada1 admin admin] The token cache is being invalidate because project 10a0a3cb403a46f08a74a5ea71558be1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:28.268451 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6123e05b-74d7-4d00-803c-9795e7a5ada1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:28.271361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6123e05b-74d7-4d00-803c-9795e7a5ada1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:28.272948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 932/1841] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:27 2022] DELETE /identity/v3/projects/10a0a3cb403a46f08a74a5ea71558be1 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:28.278833 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecdcb7-ef4f-4cc1-92de-c4b6554ac5a9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:28.279120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecdcb7-ef4f-4cc1-92de-c4b6554ac5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:28.279408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecdcb7-ef4f-4cc1-92de-c4b6554ac5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:28.293688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b36d5d3903f485a993fa0a934acf9e9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:28.312466 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b7ecdcb7-ef4f-4cc1-92de-c4b6554ac5a9 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:28.313742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b7ecdcb7-ef4f-4cc1-92de-c4b6554ac5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:28.385048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 933/1842] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:28.393841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daf6da-a779-4b9c-aa87-1053d9abb578 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:28.394130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daf6da-a779-4b9c-aa87-1053d9abb578 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:28.394405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daf6da-a779-4b9c-aa87-1053d9abb578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:28.416833 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-39daf6da-a779-4b9c-aa87-1053d9abb578 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:28.417804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-39daf6da-a779-4b9c-aa87-1053d9abb578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:28.428619 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 934/1843] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:28.433855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39daf6da-a779-4b9c-aa87-1053d9abb578 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:28.445302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39daf6da-a779-4b9c-aa87-1053d9abb578 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:28.450842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67059e3b-e724-4e65-ab7f-f163d34851c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:28.451483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67059e3b-e724-4e65-ab7f-f163d34851c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:28.451767 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67059e3b-e724-4e65-ab7f-f163d34851c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:28.452073 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67059e3b-e724-4e65-ab7f-f163d34851c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:28.452782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67059e3b-e724-4e65-ab7f-f163d34851c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:28.453092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67059e3b-e724-4e65-ab7f-f163d34851c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:28.454979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67059e3b-e724-4e65-ab7f-f163d34851c3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:28.478873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 935/1844] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:49:28.483559 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67059e3b-e724-4e65-ab7f-f163d34851c3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:28.494351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67059e3b-e724-4e65-ab7f-f163d34851c3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:28.499996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd412d-fca2-4a73-8b14-6c639ee5b515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:28.500629 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd412d-fca2-4a73-8b14-6c639ee5b515 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:28.500918 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd412d-fca2-4a73-8b14-6c639ee5b515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:28.501195 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd412d-fca2-4a73-8b14-6c639ee5b515 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:28.501877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd412d-fca2-4a73-8b14-6c639ee5b515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:28.505327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd412d-fca2-4a73-8b14-6c639ee5b515 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:28.651485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b36d5d3903f485a993fa0a934acf9e9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:28.652189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b36d5d3903f485a993fa0a934acf9e9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:28.750036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b36d5d3903f485a993fa0a934acf9e9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:28.750036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b36d5d3903f485a993fa0a934acf9e9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:28.750036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 910/1845] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:27 2022] DELETE /identity/v3/users/3b36d5d3903f485a993fa0a934acf9e9 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:29.521075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 936/1846] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:28 2022] POST /identity/v3/projects => generated 383 bytes in 1038 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:29.525694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:29.527302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:29.615227 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab8d8f-fe4a-489b-a941-269f2fc0416a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:29.640095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:29.690998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4aa26-ac36-4201-a7d4-8a440a7f6159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:29.691580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4aa26-ac36-4201-a7d4-8a440a7f6159 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:29.691697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4aa26-ac36-4201-a7d4-8a440a7f6159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:29.691950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4aa26-ac36-4201-a7d4-8a440a7f6159 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:29.692578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4aa26-ac36-4201-a7d4-8a440a7f6159 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:29.694771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4aa26-ac36-4201-a7d4-8a440a7f6159 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:29.817715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 911/1847] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:29 2022] POST /identity/v3/users => generated 292 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:29.822044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd412d-fca2-4a73-8b14-6c639ee5b515 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:29.832457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd412d-fca2-4a73-8b14-6c639ee5b515 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:29.837127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee1cdc-f75b-4939-8a02-f410f739e187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:29.837783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee1cdc-f75b-4939-8a02-f410f739e187 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:29.838113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee1cdc-f75b-4939-8a02-f410f739e187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:29.838390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee1cdc-f75b-4939-8a02-f410f739e187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:29.839035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee1cdc-f75b-4939-8a02-f410f739e187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:29.840666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee1cdc-f75b-4939-8a02-f410f739e187 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:29.851975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 937/1848] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:29.856122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4aa26-ac36-4201-a7d4-8a440a7f6159 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:29.867535 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4aa26-ac36-4201-a7d4-8a440a7f6159 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:29.870788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee1cdc-f75b-4939-8a02-f410f739e187 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:29.872644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d21fa-cf6a-48ca-8ab0-3e85f5d9dbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:29.872785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7bee1cdc-f75b-4939-8a02-f410f739e187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:29.873261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d21fa-cf6a-48ca-8ab0-3e85f5d9dbe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:29.877063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d21fa-cf6a-48ca-8ab0-3e85f5d9dbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:29.877063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d21fa-cf6a-48ca-8ab0-3e85f5d9dbe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:29.877063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d21fa-cf6a-48ca-8ab0-3e85f5d9dbe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:29.877063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d21fa-cf6a-48ca-8ab0-3e85f5d9dbe1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:29.887249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 912/1849] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:29.887410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee1cdc-f75b-4939-8a02-f410f739e187 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:29.891714 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d21fa-cf6a-48ca-8ab0-3e85f5d9dbe1 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:29.894293 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eee1a-a039-48bd-8045-e6f779143202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:29.894963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eee1a-a039-48bd-8045-e6f779143202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:29.895248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eee1a-a039-48bd-8045-e6f779143202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:29.895531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eee1a-a039-48bd-8045-e6f779143202 admin admin] PATH_INFO: `/v3/projects/4ecd3b87899a43b3b8d35c428960a41c` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:29.896692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9eee1a-a039-48bd-8045-e6f779143202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ecd3b87899a43b3b8d35c428960a41c)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:29.898962 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9eee1a-a039-48bd-8045-e6f779143202 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:29.903078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d21fa-cf6a-48ca-8ab0-3e85f5d9dbe1 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:29.908754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7cedf-63e9-4c11-9073-9e6f093e3ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:29.909386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7cedf-63e9-4c11-9073-9e6f093e3ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:29.909737 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7cedf-63e9-4c11-9073-9e6f093e3ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:29.910004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7cedf-63e9-4c11-9073-9e6f093e3ce8 admin admin] PATH_INFO: `/v3/projects/f2e510146497421591ae8fadba454784/users/02acbc4209ad44d681c90035d04147a0/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:29.936702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7cedf-63e9-4c11-9073-9e6f093e3ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2e510146497421591ae8fadba454784, user_id=02acbc4209ad44d681c90035d04147a0, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:29.942558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7cedf-63e9-4c11-9073-9e6f093e3ce8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:30.017961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 913/1850] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:29 2022] PUT /identity/v3/projects/f2e510146497421591ae8fadba454784/users/02acbc4209ad44d681c90035d04147a0/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:30.025652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3a9eee1a-a039-48bd-8045-e6f779143202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ecd3b87899a43b3b8d35c428960a41c', 'request_id': 'req-3a9eee1a-a039-48bd-8045-e6f779143202'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:30.028679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd12fc2-e549-4884-a776-2b8a4f5cdcc6 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:30.028976 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd12fc2-e549-4884-a776-2b8a4f5cdcc6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:30.029256 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd12fc2-e549-4884-a776-2b8a4f5cdcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:30.031388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3a9eee1a-a039-48bd-8045-e6f779143202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ecd3b87899a43b3b8d35c428960a41c', 'request_id': 'req-3a9eee1a-a039-48bd-8045-e6f779143202'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:30.038902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3a9eee1a-a039-48bd-8045-e6f779143202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ecd3b87899a43b3b8d35c428960a41c', 'request_id': 'req-3a9eee1a-a039-48bd-8045-e6f779143202'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:30.061940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1fd12fc2-e549-4884-a776-2b8a4f5cdcc6 None None] MFA Rules not processed for user `02acbc4209ad44d681c90035d04147a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:30.063188 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1fd12fc2-e549-4884-a776-2b8a4f5cdcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:30.119208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:30.155399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 914/1851] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:30.160060 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd12fc2-e549-4884-a776-2b8a4f5cdcc6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:30.160743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3a9eee1a-a039-48bd-8045-e6f779143202 admin admin] The token cache is being invalidate because project 4ecd3b87899a43b3b8d35c428960a41c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:30.161064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3a9eee1a-a039-48bd-8045-e6f779143202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:30.162161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3a9eee1a-a039-48bd-8045-e6f779143202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:30.164781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 938/1852] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:29 2022] DELETE /identity/v3/projects/4ecd3b87899a43b3b8d35c428960a41c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:30.176776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6844d-ac8d-4ab5-a012-3e43120a667e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:30.177282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6844d-ac8d-4ab5-a012-3e43120a667e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:30.178264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6844d-ac8d-4ab5-a012-3e43120a667e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:30.213791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-50f6844d-ac8d-4ab5-a012-3e43120a667e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:30.242088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:30.255578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-50f6844d-ac8d-4ab5-a012-3e43120a667e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:30.305021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd12fc2-e549-4884-a776-2b8a4f5cdcc6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:30.313068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c982be-01bc-45c1-a4ea-a05a0807e54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:30.314506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c982be-01bc-45c1-a4ea-a05a0807e54c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:30.314912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c982be-01bc-45c1-a4ea-a05a0807e54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:30.315284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c982be-01bc-45c1-a4ea-a05a0807e54c admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:30.316070 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c982be-01bc-45c1-a4ea-a05a0807e54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:30.318761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c982be-01bc-45c1-a4ea-a05a0807e54c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:30.397702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 939/1853] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:30.411318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69633cef-1a24-4a96-a221-348b053860be None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:30.412037 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69633cef-1a24-4a96-a221-348b053860be None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:30.412469 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69633cef-1a24-4a96-a221-348b053860be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:30.417117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 915/1854] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:30 2022] POST /identity/v3/projects => generated 383 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:30.417343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c982be-01bc-45c1-a4ea-a05a0807e54c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:30.419270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-55c982be-01bc-45c1-a4ea-a05a0807e54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:30.448135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-69633cef-1a24-4a96-a221-348b053860be None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:30.449758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-69633cef-1a24-4a96-a221-348b053860be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:30.530541 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c982be-01bc-45c1-a4ea-a05a0807e54c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:30.537319 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05636780-75ac-43b6-a505-fd6445066968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:30.538039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05636780-75ac-43b6-a505-fd6445066968 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:30.538456 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05636780-75ac-43b6-a505-fd6445066968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:30.538735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05636780-75ac-43b6-a505-fd6445066968 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:30.539436 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05636780-75ac-43b6-a505-fd6445066968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:30.541478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05636780-75ac-43b6-a505-fd6445066968 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:30.594614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 940/1855] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:30.596578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69633cef-1a24-4a96-a221-348b053860be None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:30.627585 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69633cef-1a24-4a96-a221-348b053860be None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:30.648020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf0c9e-ea9c-4b27-b014-236ec55ff3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:30.648656 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf0c9e-ea9c-4b27-b014-236ec55ff3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:30.648944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf0c9e-ea9c-4b27-b014-236ec55ff3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:30.649871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf0c9e-ea9c-4b27-b014-236ec55ff3c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:30.655077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf0c9e-ea9c-4b27-b014-236ec55ff3c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:30.655077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf0c9e-ea9c-4b27-b014-236ec55ff3c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:30.657300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf0c9e-ea9c-4b27-b014-236ec55ff3c1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:30.686829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 916/1856] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:30 2022] POST /identity/v3/users => generated 298 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:30.692050 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 941/1857] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:30.692289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05636780-75ac-43b6-a505-fd6445066968 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:30.696688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf0c9e-ea9c-4b27-b014-236ec55ff3c1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:30.707851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05636780-75ac-43b6-a505-fd6445066968 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:30.726371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718f17e-1212-4956-bc00-bf26b70ce9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:30.726904 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf0c9e-ea9c-4b27-b014-236ec55ff3c1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:30.727219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718f17e-1212-4956-bc00-bf26b70ce9c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:30.727569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718f17e-1212-4956-bc00-bf26b70ce9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:30.727919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718f17e-1212-4956-bc00-bf26b70ce9c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:30.728740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718f17e-1212-4956-bc00-bf26b70ce9c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:30.730974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718f17e-1212-4956-bc00-bf26b70ce9c2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:30.737209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b0631-a992-4641-9384-ed658ce49444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:30.738412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b0631-a992-4641-9384-ed658ce49444 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:30.738412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b0631-a992-4641-9384-ed658ce49444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:30.738632 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b0631-a992-4641-9384-ed658ce49444 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:30.739336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b0631-a992-4641-9384-ed658ce49444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:30.741317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b0631-a992-4641-9384-ed658ce49444 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:30.750316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 917/1858] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:49:30.752511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718f17e-1212-4956-bc00-bf26b70ce9c2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:30.767270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718f17e-1212-4956-bc00-bf26b70ce9c2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:30.774558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89f208-210d-4141-851a-cb2dd4ff5b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:30.776069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f208-210d-4141-851a-cb2dd4ff5b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:30.776363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f208-210d-4141-851a-cb2dd4ff5b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:30.776644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f208-210d-4141-851a-cb2dd4ff5b3d admin admin] PATH_INFO: `/v3/projects/9e9b6e0090fd44caaa0daba325ac8c1d/users/b101b6113f344b6a81eebf5943fe6093/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:30.809102 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89f208-210d-4141-851a-cb2dd4ff5b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9b6e0090fd44caaa0daba325ac8c1d, user_id=b101b6113f344b6a81eebf5943fe6093, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:30.811841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89f208-210d-4141-851a-cb2dd4ff5b3d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:30.929621 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 942/1859] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:30 2022] POST /identity/v3/projects => generated 391 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:30.931029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b0631-a992-4641-9384-ed658ce49444 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:30.967787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 918/1860] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:30 2022] PUT /identity/v3/projects/9e9b6e0090fd44caaa0daba325ac8c1d/users/b101b6113f344b6a81eebf5943fe6093/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:30.976197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89f208-210d-4141-851a-cb2dd4ff5b3d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:31.042414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b0631-a992-4641-9384-ed658ce49444 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:31.100884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89f208-210d-4141-851a-cb2dd4ff5b3d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:31.106842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214687e-10d7-4190-8500-627ab32f62f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:31.107501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214687e-10d7-4190-8500-627ab32f62f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:31.107782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214687e-10d7-4190-8500-627ab32f62f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:31.108057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214687e-10d7-4190-8500-627ab32f62f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:31.108710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214687e-10d7-4190-8500-627ab32f62f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:31.112714 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214687e-10d7-4190-8500-627ab32f62f8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:31.130664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 919/1861] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:31.138603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214687e-10d7-4190-8500-627ab32f62f8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:31.148005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8afd77-5e03-40ef-a548-1d7f93465ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:31.148315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8afd77-5e03-40ef-a548-1d7f93465ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:31.148583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8afd77-5e03-40ef-a548-1d7f93465ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:31.153793 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8afd77-5e03-40ef-a548-1d7f93465ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:31.154529 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8afd77-5e03-40ef-a548-1d7f93465ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:31.158265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8afd77-5e03-40ef-a548-1d7f93465ec7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:31.163572 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214687e-10d7-4190-8500-627ab32f62f8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:31.169212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0d843-f0cc-42c8-8517-45d606c581ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:31.177475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d843-f0cc-42c8-8517-45d606c581ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:31.177475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d843-f0cc-42c8-8517-45d606c581ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:31.177475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d843-f0cc-42c8-8517-45d606c581ac admin admin] PATH_INFO: `/v3/projects/9e9b6e0090fd44caaa0daba325ac8c1d/users/b101b6113f344b6a81eebf5943fe6093/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:31.177475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0d843-f0cc-42c8-8517-45d606c581ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9b6e0090fd44caaa0daba325ac8c1d, user_id=b101b6113f344b6a81eebf5943fe6093, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:31.177475 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0d843-f0cc-42c8-8517-45d606c581ac admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:31.189591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 920/1862] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:31 2022] PUT /identity/v3/projects/9e9b6e0090fd44caaa0daba325ac8c1d/users/b101b6113f344b6a81eebf5943fe6093/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:31.200536 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97440d89-aab2-49a6-85e7-f1032d8b62f6 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:31.200877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97440d89-aab2-49a6-85e7-f1032d8b62f6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:31.201108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97440d89-aab2-49a6-85e7-f1032d8b62f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:31.243954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-97440d89-aab2-49a6-85e7-f1032d8b62f6 None None] MFA Rules not processed for user `b101b6113f344b6a81eebf5943fe6093`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:31.244694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-97440d89-aab2-49a6-85e7-f1032d8b62f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:31.347942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 921/1863] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:31.361559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97440d89-aab2-49a6-85e7-f1032d8b62f6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:31.365576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 943/1864] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:30 2022] POST /identity/v3/users => generated 296 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:31.365740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-97440d89-aab2-49a6-85e7-f1032d8b62f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:31.366721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8afd77-5e03-40ef-a548-1d7f93465ec7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:31.378245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8afd77-5e03-40ef-a548-1d7f93465ec7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:31.450816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d40540-0d6c-435f-a219-c6689cbf5d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:31.451219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d40540-0d6c-435f-a219-c6689cbf5d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:31.451501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d40540-0d6c-435f-a219-c6689cbf5d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:31.451824 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d40540-0d6c-435f-a219-c6689cbf5d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:31.452453 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d40540-0d6c-435f-a219-c6689cbf5d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:31.454420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d40540-0d6c-435f-a219-c6689cbf5d1f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:31.464507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97440d89-aab2-49a6-85e7-f1032d8b62f6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:31.465325 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 944/1865] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:31.469116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d40540-0d6c-435f-a219-c6689cbf5d1f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:31.469269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a82604-d9bf-4f2c-81bf-d45a6240e341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:31.470624 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a82604-d9bf-4f2c-81bf-d45a6240e341 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:31.470909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a82604-d9bf-4f2c-81bf-d45a6240e341 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:31.471181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a82604-d9bf-4f2c-81bf-d45a6240e341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:31.481511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d40540-0d6c-435f-a219-c6689cbf5d1f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:31.487016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a82604-d9bf-4f2c-81bf-d45a6240e341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:31.487489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e49d8-c07a-4bf5-81a7-a2390d2d3ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:31.488278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e49d8-c07a-4bf5-81a7-a2390d2d3ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:31.488566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e49d8-c07a-4bf5-81a7-a2390d2d3ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:31.488670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a82604-d9bf-4f2c-81bf-d45a6240e341 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:31.488843 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e49d8-c07a-4bf5-81a7-a2390d2d3ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:31.489495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e49d8-c07a-4bf5-81a7-a2390d2d3ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:31.491119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e49d8-c07a-4bf5-81a7-a2390d2d3ff9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:31.499165 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 922/1866] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:31.506720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 945/1867] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:49:31.510910 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a82604-d9bf-4f2c-81bf-d45a6240e341 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:31.521779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a82604-d9bf-4f2c-81bf-d45a6240e341 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:31.526778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2cdd0-26f5-494a-88e7-d55796d1456a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:31.527432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2cdd0-26f5-494a-88e7-d55796d1456a admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:31.527710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2cdd0-26f5-494a-88e7-d55796d1456a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:31.528012 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2cdd0-26f5-494a-88e7-d55796d1456a admin admin] PATH_INFO: `/v3/projects/f135df8c000f4a31a4c65e6c2574abcc/users/aa153692a4ea4b228da3bf1ed2853b57/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:31.546457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2cdd0-26f5-494a-88e7-d55796d1456a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f135df8c000f4a31a4c65e6c2574abcc, user_id=aa153692a4ea4b228da3bf1ed2853b57, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:31.550234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2cdd0-26f5-494a-88e7-d55796d1456a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:31.652734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 923/1868] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:31 2022] PUT /identity/v3/projects/f135df8c000f4a31a4c65e6c2574abcc/users/aa153692a4ea4b228da3bf1ed2853b57/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:31.666329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd167a97-91da-4daf-a2e7-0b205c071045 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:31.666960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd167a97-91da-4daf-a2e7-0b205c071045 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:31.667119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd167a97-91da-4daf-a2e7-0b205c071045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:31.700531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-dd167a97-91da-4daf-a2e7-0b205c071045 None None] MFA Rules not processed for user `aa153692a4ea4b228da3bf1ed2853b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:31.700531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dd167a97-91da-4daf-a2e7-0b205c071045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:31.780740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 946/1869] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:31.780935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2cdd0-26f5-494a-88e7-d55796d1456a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:31.846869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd167a97-91da-4daf-a2e7-0b205c071045 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:31.860492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2cdd0-26f5-494a-88e7-d55796d1456a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:31.872158 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2eb0f-cd20-4065-b5dd-b8234bcf4086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:31.872158 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2eb0f-cd20-4065-b5dd-b8234bcf4086 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:31.872158 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2eb0f-cd20-4065-b5dd-b8234bcf4086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:31.872158 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2eb0f-cd20-4065-b5dd-b8234bcf4086 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:31.872550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2eb0f-cd20-4065-b5dd-b8234bcf4086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:31.872550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2eb0f-cd20-4065-b5dd-b8234bcf4086 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:31.911461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd167a97-91da-4daf-a2e7-0b205c071045 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:31.915179 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfa81c-8a4d-4f01-9868-7f1da939aed8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:31.915834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfa81c-8a4d-4f01-9868-7f1da939aed8 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:31.916114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfa81c-8a4d-4f01-9868-7f1da939aed8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:31.916387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfa81c-8a4d-4f01-9868-7f1da939aed8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:31.918222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-23dfa81c-8a4d-4f01-9868-7f1da939aed8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:31.987394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 924/1870] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:31 2022] POST /identity/v3/projects => generated 393 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:31.994856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2eb0f-cd20-4065-b5dd-b8234bcf4086 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:32.020162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfa81c-8a4d-4f01-9868-7f1da939aed8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:32.022655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfa81c-8a4d-4f01-9868-7f1da939aed8 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:32.039873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 947/1871] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:32.210322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2eb0f-cd20-4065-b5dd-b8234bcf4086 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:32.210322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f7a97-e757-41c2-ad5a-ef1c3d249192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:32.210322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f7a97-e757-41c2-ad5a-ef1c3d249192 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:32.210322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f7a97-e757-41c2-ad5a-ef1c3d249192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:32.210689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f7a97-e757-41c2-ad5a-ef1c3d249192 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:32.210689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f7a97-e757-41c2-ad5a-ef1c3d249192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:32.210689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f7a97-e757-41c2-ad5a-ef1c3d249192 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:32.437370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 925/1872] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:31 2022] POST /identity/v3/users => generated 303 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:32.442148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfa81c-8a4d-4f01-9868-7f1da939aed8 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:32.459428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfa81c-8a4d-4f01-9868-7f1da939aed8 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:32.473125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cb163-07e5-4998-a1c1-d15da9c4e6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:32.473125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cb163-07e5-4998-a1c1-d15da9c4e6ea admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:32.473125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cb163-07e5-4998-a1c1-d15da9c4e6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:32.473125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cb163-07e5-4998-a1c1-d15da9c4e6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:32.473484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cb163-07e5-4998-a1c1-d15da9c4e6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:32.473484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cb163-07e5-4998-a1c1-d15da9c4e6ea admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:32.492891 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 948/1873] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:32.497485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f7a97-e757-41c2-ad5a-ef1c3d249192 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:32.518581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f7a97-e757-41c2-ad5a-ef1c3d249192 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:32.522991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ef93a-2f18-41f1-8083-c52b4a3c4af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:32.524217 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ef93a-2f18-41f1-8083-c52b4a3c4af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:32.524621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ef93a-2f18-41f1-8083-c52b4a3c4af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:32.524974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ef93a-2f18-41f1-8083-c52b4a3c4af8 admin admin] PATH_INFO: `/v3/projects/4fa69aa874ef46ef9359383bdf4e4c37/users/106bbf91d2914760a58e7625d8249bdd/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:32.547350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ef93a-2f18-41f1-8083-c52b4a3c4af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa69aa874ef46ef9359383bdf4e4c37, user_id=106bbf91d2914760a58e7625d8249bdd, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:32.551534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ef93a-2f18-41f1-8083-c52b4a3c4af8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:32.659339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:32.665625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 926/1874] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:32 2022] PUT /identity/v3/projects/4fa69aa874ef46ef9359383bdf4e4c37/users/106bbf91d2914760a58e7625d8249bdd/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:32.670444 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cb163-07e5-4998-a1c1-d15da9c4e6ea admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:32.679740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf261f-1d70-400f-ac4b-6977e26b3bf4 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:32.680039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf261f-1d70-400f-ac4b-6977e26b3bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:32.680320 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf261f-1d70-400f-ac4b-6977e26b3bf4 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:32.682121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 927/1875] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:49:32.684910 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cb163-07e5-4998-a1c1-d15da9c4e6ea admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:32.748447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60453ef7-109c-4c13-9daa-69ea23207628 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:32.748783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60453ef7-109c-4c13-9daa-69ea23207628 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:32.749024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60453ef7-109c-4c13-9daa-69ea23207628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:32.755256 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edcae9-27d1-4a2b-add8-fbd9e798caca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:32.755910 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edcae9-27d1-4a2b-add8-fbd9e798caca admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:32.756191 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edcae9-27d1-4a2b-add8-fbd9e798caca admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:32.756470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edcae9-27d1-4a2b-add8-fbd9e798caca admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:32.757270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edcae9-27d1-4a2b-add8-fbd9e798caca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:32.759059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edcae9-27d1-4a2b-add8-fbd9e798caca admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:32.774469 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 949/1876] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:32.781962 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edcae9-27d1-4a2b-add8-fbd9e798caca admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:32.797567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-60453ef7-109c-4c13-9daa-69ea23207628 None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:32.799324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edcae9-27d1-4a2b-add8-fbd9e798caca admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:32.810643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-60453ef7-109c-4c13-9daa-69ea23207628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:32.816341 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15621494-1684-4c32-a5d0-97480b206e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:32.827984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15621494-1684-4c32-a5d0-97480b206e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:32.827984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15621494-1684-4c32-a5d0-97480b206e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:32.827984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15621494-1684-4c32-a5d0-97480b206e9e admin admin] PATH_INFO: `/v3/projects/4fa69aa874ef46ef9359383bdf4e4c37/users/106bbf91d2914760a58e7625d8249bdd/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:32.827984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15621494-1684-4c32-a5d0-97480b206e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa69aa874ef46ef9359383bdf4e4c37, user_id=106bbf91d2914760a58e7625d8249bdd, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:32.828909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15621494-1684-4c32-a5d0-97480b206e9e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:32.846741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 950/1877] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:32 2022] PUT /identity/v3/projects/4fa69aa874ef46ef9359383bdf4e4c37/users/106bbf91d2914760a58e7625d8249bdd/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:32.866357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faf7bc-8fc1-4aac-8d2f-1450a6b14fc0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:32.866990 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faf7bc-8fc1-4aac-8d2f-1450a6b14fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:32.867301 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faf7bc-8fc1-4aac-8d2f-1450a6b14fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:32.904310 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-87faf7bc-8fc1-4aac-8d2f-1450a6b14fc0 None None] MFA Rules not processed for user `106bbf91d2914760a58e7625d8249bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:32.905735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-87faf7bc-8fc1-4aac-8d2f-1450a6b14fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:32.910808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 928/1878] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:49:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:32.931219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60453ef7-109c-4c13-9daa-69ea23207628 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:32.933148 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-60453ef7-109c-4c13-9daa-69ea23207628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:32.954394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60453ef7-109c-4c13-9daa-69ea23207628 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:32.992298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 951/1879] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:32 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:33.025647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9a9ac-8497-44a7-ba54-3bd3a959e80f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:33.026347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9a9ac-8497-44a7-ba54-3bd3a959e80f service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:33.026696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9a9ac-8497-44a7-ba54-3bd3a959e80f service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:33.027037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9a9ac-8497-44a7-ba54-3bd3a959e80f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:33.038220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9a9ac-8497-44a7-ba54-3bd3a959e80f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:33.040294 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9a9ac-8497-44a7-ba54-3bd3a959e80f service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:33.063046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 929/1880] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:33.171847 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:33.264601 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faf7bc-8fc1-4aac-8d2f-1450a6b14fc0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:33.321573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faf7bc-8fc1-4aac-8d2f-1450a6b14fc0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:33.326074 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a3a26-3b21-4086-82ea-b1a5896bc4e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:33.326681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a3a26-3b21-4086-82ea-b1a5896bc4e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:33.326950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a3a26-3b21-4086-82ea-b1a5896bc4e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:33.327267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a3a26-3b21-4086-82ea-b1a5896bc4e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:33.337203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a3a26-3b21-4086-82ea-b1a5896bc4e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:33.338674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a3a26-3b21-4086-82ea-b1a5896bc4e0 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:33.349398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 952/1881] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:33 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:33.401378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9a9ac-8497-44a7-ba54-3bd3a959e80f service nova] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:33.529086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9a9ac-8497-44a7-ba54-3bd3a959e80f service nova] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:33.538897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9a225-16a5-45fb-9632-dce561f1f59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:33.555686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9a225-16a5-45fb-9632-dce561f1f59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:33.555686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9a225-16a5-45fb-9632-dce561f1f59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:33.555686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9a225-16a5-45fb-9632-dce561f1f59e admin admin] PATH_INFO: `/v3/users/02acbc4209ad44d681c90035d04147a0` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:33.555686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9a225-16a5-45fb-9632-dce561f1f59e admin admin] RBAC: Authorizing `identity:delete_user(user_id=02acbc4209ad44d681c90035d04147a0)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:33.555686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9a225-16a5-45fb-9632-dce561f1f59e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:33.919013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-dcb9a225-16a5-45fb-9632-dce561f1f59e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02acbc4209ad44d681c90035d04147a0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:34.069749 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-dcb9a225-16a5-45fb-9632-dce561f1f59e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02acbc4209ad44d681c90035d04147a0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:34.070387 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-dcb9a225-16a5-45fb-9632-dce561f1f59e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02acbc4209ad44d681c90035d04147a0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:34.084391 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-dcb9a225-16a5-45fb-9632-dce561f1f59e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02acbc4209ad44d681c90035d04147a0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:34.088929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a3a26-3b21-4086-82ea-b1a5896bc4e0 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:34.098570 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a3a26-3b21-4086-82ea-b1a5896bc4e0 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:34.103021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14ea13-b799-4c05-9a9f-a75cbdd86b80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:34.103777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-dcb9a225-16a5-45fb-9632-dce561f1f59e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02acbc4209ad44d681c90035d04147a0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:34.103897 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14ea13-b799-4c05-9a9f-a75cbdd86b80 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:34.104134 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14ea13-b799-4c05-9a9f-a75cbdd86b80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:34.104417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14ea13-b799-4c05-9a9f-a75cbdd86b80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:34.107615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ab14ea13-b799-4c05-9a9f-a75cbdd86b80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:34.111865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 930/1882] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:33 2022] DELETE /identity/v3/users/02acbc4209ad44d681c90035d04147a0 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:34.116474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17a2a4-b459-4f6c-9736-148391326f89 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:34.116633 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17a2a4-b459-4f6c-9736-148391326f89 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:34.116988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17a2a4-b459-4f6c-9736-148391326f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:34.141652 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1c17a2a4-b459-4f6c-9736-148391326f89 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:34.142885 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1c17a2a4-b459-4f6c-9736-148391326f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:34.174554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14ea13-b799-4c05-9a9f-a75cbdd86b80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:34.182451 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14ea13-b799-4c05-9a9f-a75cbdd86b80 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:34.187096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 953/1883] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:34.205195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 931/1884] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:34.215480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14ea13-b799-4c05-9a9f-a75cbdd86b80 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:34.217194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ab14ea13-b799-4c05-9a9f-a75cbdd86b80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:34.311215 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14ea13-b799-4c05-9a9f-a75cbdd86b80 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:34.372369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a38d0e-f243-48e2-9456-01c9fabf6575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:34.372987 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a38d0e-f243-48e2-9456-01c9fabf6575 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:34.373254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a38d0e-f243-48e2-9456-01c9fabf6575 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:34.373556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a38d0e-f243-48e2-9456-01c9fabf6575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:34.383856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a38d0e-f243-48e2-9456-01c9fabf6575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:34.385316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a38d0e-f243-48e2-9456-01c9fabf6575 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:34.395779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 954/1885] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:34.608282 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17a2a4-b459-4f6c-9736-148391326f89 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:34.609741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1c17a2a4-b459-4f6c-9736-148391326f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:34.624692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17a2a4-b459-4f6c-9736-148391326f89 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:34.629919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:34.630664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:34.631013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:34.631357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] PATH_INFO: `/v3/users/04c2f8262a1b4f2383063cc9497d157b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:34.632923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c2f8262a1b4f2383063cc9497d157b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:34.634961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:34.959870 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c2f8262a1b4f2383063cc9497d157b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:35.112485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c2f8262a1b4f2383063cc9497d157b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:35.112485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c2f8262a1b4f2383063cc9497d157b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:35.119954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c2f8262a1b4f2383063cc9497d157b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:35.142853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c2f8262a1b4f2383063cc9497d157b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:35.148103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 932/1886] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:34 2022] DELETE /identity/v3/users/04c2f8262a1b4f2383063cc9497d157b => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:35.153171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04355064-d4b9-4c04-ab5b-53bf80a0cea4 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:35.153171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04355064-d4b9-4c04-ab5b-53bf80a0cea4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:35.153809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04355064-d4b9-4c04-ab5b-53bf80a0cea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:35.180421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-04355064-d4b9-4c04-ab5b-53bf80a0cea4 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:35.181821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-04355064-d4b9-4c04-ab5b-53bf80a0cea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:35.278331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 955/1887] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:35.292979 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:35.294528 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:35.357348 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7da04d-35f5-4a8e-9eb3-00e37e7588a3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:35.363252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f51d40-83c8-4b74-845f-72539ce7756a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:35.364928 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f51d40-83c8-4b74-845f-72539ce7756a service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:35.367231 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04355064-d4b9-4c04-ab5b-53bf80a0cea4 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:35.368695 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-04355064-d4b9-4c04-ab5b-53bf80a0cea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:35.370776 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f51d40-83c8-4b74-845f-72539ce7756a service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:35.371177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f51d40-83c8-4b74-845f-72539ce7756a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:35.384342 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f51d40-83c8-4b74-845f-72539ce7756a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:35.386092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f51d40-83c8-4b74-845f-72539ce7756a service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:35.392586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04355064-d4b9-4c04-ab5b-53bf80a0cea4 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:35.398709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:35.404949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:35.404949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:35.404949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] PATH_INFO: `/v3/projects/f2e510146497421591ae8fadba454784` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:35.406374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 933/1888] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:35.409502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2e510146497421591ae8fadba454784)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:35.411596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:35.646298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2e510146497421591ae8fadba454784', 'request_id': 'req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:35.651556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2e510146497421591ae8fadba454784', 'request_id': 'req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:35.657801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2e510146497421591ae8fadba454784', 'request_id': 'req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:35.748106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] The token cache is being invalidate because project f2e510146497421591ae8fadba454784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:35.748525 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:35.749172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:35.750746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 956/1889] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:35 2022] DELETE /identity/v3/projects/f2e510146497421591ae8fadba454784 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:35.755529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f51d40-83c8-4b74-845f-72539ce7756a service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:35.765325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f51d40-83c8-4b74-845f-72539ce7756a service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:35.826530 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:35.827179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:35.827467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:35.827740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] PATH_INFO: `/v3/users/b101b6113f344b6a81eebf5943fe6093` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:35.829153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b101b6113f344b6a81eebf5943fe6093)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:35.831248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:35.995671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:35.997315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:36.014379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47124-b9e2-4b9b-a8ed-a6bc0525518e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:36.022955 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:36.027523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:36.027523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:36.027523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] PATH_INFO: `/v3/projects/3907f1c5094b4cb68464311e544f11c1` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:36.027523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3907f1c5094b4cb68464311e544f11c1)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:36.027523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:36.198596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b101b6113f344b6a81eebf5943fe6093'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.201652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3907f1c5094b4cb68464311e544f11c1', 'request_id': 'req-14d9cd78-587c-4438-a69c-42cdd98c4080'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.208080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3907f1c5094b4cb68464311e544f11c1', 'request_id': 'req-14d9cd78-587c-4438-a69c-42cdd98c4080'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.214785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3907f1c5094b4cb68464311e544f11c1', 'request_id': 'req-14d9cd78-587c-4438-a69c-42cdd98c4080'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.336941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b101b6113f344b6a81eebf5943fe6093'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.337766 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b101b6113f344b6a81eebf5943fe6093'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.346264 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b101b6113f344b6a81eebf5943fe6093'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.367151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b101b6113f344b6a81eebf5943fe6093'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.384004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 934/1890] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:35 2022] DELETE /identity/v3/users/b101b6113f344b6a81eebf5943fe6093 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:36.467964 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] The token cache is being invalidate because project 3907f1c5094b4cb68464311e544f11c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:36.468484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.469319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.471980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 957/1891] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:35 2022] DELETE /identity/v3/projects/3907f1c5094b4cb68464311e544f11c1 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:36.477426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:36.479600 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:36.553206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd781bf-c485-4d2e-9f4b-47c5c6cd14e1 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:36.558400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed826b-06e9-4e71-95e2-fcf1b47dbd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:36.559082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed826b-06e9-4e71-95e2-fcf1b47dbd8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:36.562444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed826b-06e9-4e71-95e2-fcf1b47dbd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:36.562444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed826b-06e9-4e71-95e2-fcf1b47dbd8d admin admin] PATH_INFO: `/v3/users/deeee1670f774ed8afa93e77c8a48c3c` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:36.562444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed826b-06e9-4e71-95e2-fcf1b47dbd8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=deeee1670f774ed8afa93e77c8a48c3c)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:36.562739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed826b-06e9-4e71-95e2-fcf1b47dbd8d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:36.691590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:36.796459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9bed826b-06e9-4e71-95e2-fcf1b47dbd8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deeee1670f774ed8afa93e77c8a48c3c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.849261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:36.850047 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:36.868405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9bed826b-06e9-4e71-95e2-fcf1b47dbd8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deeee1670f774ed8afa93e77c8a48c3c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.869842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9bed826b-06e9-4e71-95e2-fcf1b47dbd8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deeee1670f774ed8afa93e77c8a48c3c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.879773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9bed826b-06e9-4e71-95e2-fcf1b47dbd8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deeee1670f774ed8afa93e77c8a48c3c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.894504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9bed826b-06e9-4e71-95e2-fcf1b47dbd8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deeee1670f774ed8afa93e77c8a48c3c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:36.901026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 935/1892] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:36 2022] DELETE /identity/v3/users/deeee1670f774ed8afa93e77c8a48c3c => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:36.937448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9cd78-587c-4438-a69c-42cdd98c4080 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:36.977247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:37.001911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897de3f-ea47-4546-abee-bbfa88059b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:37.003415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897de3f-ea47-4546-abee-bbfa88059b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:37.003701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897de3f-ea47-4546-abee-bbfa88059b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:37.004318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897de3f-ea47-4546-abee-bbfa88059b9b admin admin] PATH_INFO: `/v3/projects/9e9b6e0090fd44caaa0daba325ac8c1d` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:37.005776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897de3f-ea47-4546-abee-bbfa88059b9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e9b6e0090fd44caaa0daba325ac8c1d)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:37.007948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897de3f-ea47-4546-abee-bbfa88059b9b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:37.197117 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6897de3f-ea47-4546-abee-bbfa88059b9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e9b6e0090fd44caaa0daba325ac8c1d', 'request_id': 'req-6897de3f-ea47-4546-abee-bbfa88059b9b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:37.203845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6897de3f-ea47-4546-abee-bbfa88059b9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e9b6e0090fd44caaa0daba325ac8c1d', 'request_id': 'req-6897de3f-ea47-4546-abee-bbfa88059b9b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:37.216242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6897de3f-ea47-4546-abee-bbfa88059b9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e9b6e0090fd44caaa0daba325ac8c1d', 'request_id': 'req-6897de3f-ea47-4546-abee-bbfa88059b9b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:37.326489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6897de3f-ea47-4546-abee-bbfa88059b9b admin admin] The token cache is being invalidate because project 9e9b6e0090fd44caaa0daba325ac8c1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:37.326932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6897de3f-ea47-4546-abee-bbfa88059b9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:37.327811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-6897de3f-ea47-4546-abee-bbfa88059b9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:37.330315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 958/1893] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:36 2022] DELETE /identity/v3/projects/9e9b6e0090fd44caaa0daba325ac8c1d => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:37.337664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d867f42-7796-4a5a-8d53-f8436e1c605f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:37.339030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d867f42-7796-4a5a-8d53-f8436e1c605f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:37.339555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d867f42-7796-4a5a-8d53-f8436e1c605f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:37.368494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6d867f42-7796-4a5a-8d53-f8436e1c605f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:37.370681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6d867f42-7796-4a5a-8d53-f8436e1c605f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:37.466737 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 936/1894] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:37.479623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f40ac-f25d-4a2a-944c-d27f1db04896 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:37.479623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f40ac-f25d-4a2a-944c-d27f1db04896 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:37.479623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f40ac-f25d-4a2a-944c-d27f1db04896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:37.502126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-906f40ac-f25d-4a2a-944c-d27f1db04896 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:37.503441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-906f40ac-f25d-4a2a-944c-d27f1db04896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:37.511524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 959/1895] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:37.516229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d867f42-7796-4a5a-8d53-f8436e1c605f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:37.527564 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d867f42-7796-4a5a-8d53-f8436e1c605f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:37.532361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57286942-aba2-4637-a563-dcc32046044d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:37.532966 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57286942-aba2-4637-a563-dcc32046044d admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:37.533247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57286942-aba2-4637-a563-dcc32046044d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:37.533517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57286942-aba2-4637-a563-dcc32046044d admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:37.534234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57286942-aba2-4637-a563-dcc32046044d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:37.534537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57286942-aba2-4637-a563-dcc32046044d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:37.536328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57286942-aba2-4637-a563-dcc32046044d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:37.549436 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 937/1896] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:37.555957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f40ac-f25d-4a2a-944c-d27f1db04896 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:37.556306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57286942-aba2-4637-a563-dcc32046044d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:37.558600 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-57286942-aba2-4637-a563-dcc32046044d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:37.567107 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f40ac-f25d-4a2a-944c-d27f1db04896 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:37.578413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57286942-aba2-4637-a563-dcc32046044d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:37.587713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:37.589571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c80d7e-abf4-4827-8df1-3ce1af52df43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:37.597228 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:37.597991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c80d7e-abf4-4827-8df1-3ce1af52df43 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:37.597991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c80d7e-abf4-4827-8df1-3ce1af52df43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:37.598170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:37.598247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c80d7e-abf4-4827-8df1-3ce1af52df43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:37.598316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] PATH_INFO: `/v3/projects/50f4d55608f140e3b7782a9d09431513` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:37.598385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c80d7e-abf4-4827-8df1-3ce1af52df43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:37.598453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50f4d55608f140e3b7782a9d09431513)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:37.598453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:37.602224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c80d7e-abf4-4827-8df1-3ce1af52df43 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:37.731462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 960/1897] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:37 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:37.732593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c80d7e-abf4-4827-8df1-3ce1af52df43 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:37.742931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50f4d55608f140e3b7782a9d09431513', 'request_id': 'req-d31f414d-2491-4cd6-ad03-3c69a4766830'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:37.755299 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50f4d55608f140e3b7782a9d09431513', 'request_id': 'req-d31f414d-2491-4cd6-ad03-3c69a4766830'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:37.761374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50f4d55608f140e3b7782a9d09431513', 'request_id': 'req-d31f414d-2491-4cd6-ad03-3c69a4766830'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:37.799010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c80d7e-abf4-4827-8df1-3ce1af52df43 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:37.860997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2964c9-39c5-434d-b726-e0d47189cee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:37.861807 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2964c9-39c5-434d-b726-e0d47189cee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:37.862264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2964c9-39c5-434d-b726-e0d47189cee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:37.862701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2964c9-39c5-434d-b726-e0d47189cee4 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:37.863599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2964c9-39c5-434d-b726-e0d47189cee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:37.865565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2964c9-39c5-434d-b726-e0d47189cee4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:37.896984 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] The token cache is being invalidate because project 50f4d55608f140e3b7782a9d09431513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:37.897331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:37.898083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:37.899696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 938/1898] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:37 2022] DELETE /identity/v3/projects/50f4d55608f140e3b7782a9d09431513 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:37.903580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:37.905334 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:37.930546 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f414d-2491-4cd6-ad03-3c69a4766830 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:37.935402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:37.936103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:37.936395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:37.936670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] PATH_INFO: `/v3/users/df806c112adb4a06944b45a66b937ddd` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:37.938917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df806c112adb4a06944b45a66b937ddd)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:37.941780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:37.978764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 961/1899] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:37 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:37.983490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2964c9-39c5-434d-b726-e0d47189cee4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:37.987745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ad2964c9-39c5-434d-b726-e0d47189cee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:38.003658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2964c9-39c5-434d-b726-e0d47189cee4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:38.009110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44067ae-070d-4762-93a5-f206ab5a8cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:38.009765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44067ae-070d-4762-93a5-f206ab5a8cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:38.010049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44067ae-070d-4762-93a5-f206ab5a8cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:38.011939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44067ae-070d-4762-93a5-f206ab5a8cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:38.011939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44067ae-070d-4762-93a5-f206ab5a8cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:38.013049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44067ae-070d-4762-93a5-f206ab5a8cb3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:38.028810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 962/1900] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:49:38.033643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44067ae-070d-4762-93a5-f206ab5a8cb3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:38.045079 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44067ae-070d-4762-93a5-f206ab5a8cb3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:38.051160 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b018f-f43f-48aa-a391-fcbe16fbfa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:38.051906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b018f-f43f-48aa-a391-fcbe16fbfa63 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:38.052175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b018f-f43f-48aa-a391-fcbe16fbfa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:38.052427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b018f-f43f-48aa-a391-fcbe16fbfa63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:38.053158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b018f-f43f-48aa-a391-fcbe16fbfa63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:38.055633 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b018f-f43f-48aa-a391-fcbe16fbfa63 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:38.072132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 963/1901] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:38.073228 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b018f-f43f-48aa-a391-fcbe16fbfa63 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:38.083700 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b018f-f43f-48aa-a391-fcbe16fbfa63 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:38.097977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807d10f-8599-462f-8a14-91d75bb379ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:38.098766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807d10f-8599-462f-8a14-91d75bb379ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:38.099128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807d10f-8599-462f-8a14-91d75bb379ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:38.099476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807d10f-8599-462f-8a14-91d75bb379ef admin admin] PATH_INFO: `/v3/projects/2786e75ce58040ae96164ac504ce9ec7/users/08bf094c836a4878884a9c735121753f/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:38.116622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807d10f-8599-462f-8a14-91d75bb379ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=2786e75ce58040ae96164ac504ce9ec7, user_id=08bf094c836a4878884a9c735121753f, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:38.119389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807d10f-8599-462f-8a14-91d75bb379ef admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:38.144356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df806c112adb4a06944b45a66b937ddd'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:38.234764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 964/1902] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:38 2022] PUT /identity/v3/projects/2786e75ce58040ae96164ac504ce9ec7/users/08bf094c836a4878884a9c735121753f/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:38.265638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8361c3-fcab-435a-bdae-e02a0ca0d078 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:38.265638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8361c3-fcab-435a-bdae-e02a0ca0d078 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:38.266249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df806c112adb4a06944b45a66b937ddd'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:38.266435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8361c3-fcab-435a-bdae-e02a0ca0d078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:38.266594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df806c112adb4a06944b45a66b937ddd'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:38.271847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df806c112adb4a06944b45a66b937ddd'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:38.301383 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df806c112adb4a06944b45a66b937ddd'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:38.301669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-3b8361c3-fcab-435a-bdae-e02a0ca0d078 None None] MFA Rules not processed for user `08bf094c836a4878884a9c735121753f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:38.303120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3b8361c3-fcab-435a-bdae-e02a0ca0d078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:38.315004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 939/1903] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:37 2022] DELETE /identity/v3/users/df806c112adb4a06944b45a66b937ddd => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:38.380480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 965/1904] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:38.394208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:38.397219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:38.465145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c39c1-f91c-4b0a-981a-5951d0bea985 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:38.478422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104f6a0-4347-4a5e-856f-8306260d756a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:38.478422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104f6a0-4347-4a5e-856f-8306260d756a service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:38.478422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104f6a0-4347-4a5e-856f-8306260d756a service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:38.478422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104f6a0-4347-4a5e-856f-8306260d756a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:38.546150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104f6a0-4347-4a5e-856f-8306260d756a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:38.548227 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104f6a0-4347-4a5e-856f-8306260d756a service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:38.562399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 940/1905] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:38.971761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:39.013382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8361c3-fcab-435a-bdae-e02a0ca0d078 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:39.015022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3b8361c3-fcab-435a-bdae-e02a0ca0d078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:39.117934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8361c3-fcab-435a-bdae-e02a0ca0d078 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:39.184752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706ffc5-e6a2-4e71-96ca-21163ad68409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:39.185384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706ffc5-e6a2-4e71-96ca-21163ad68409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:39.185703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706ffc5-e6a2-4e71-96ca-21163ad68409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:39.186021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706ffc5-e6a2-4e71-96ca-21163ad68409 admin admin] PATH_INFO: `/v3/projects/308df6c13ef64ce8992f61f31abea8f7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:39.187196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706ffc5-e6a2-4e71-96ca-21163ad68409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=308df6c13ef64ce8992f61f31abea8f7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:39.189125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706ffc5-e6a2-4e71-96ca-21163ad68409 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:39.396584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2706ffc5-e6a2-4e71-96ca-21163ad68409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '308df6c13ef64ce8992f61f31abea8f7', 'request_id': 'req-2706ffc5-e6a2-4e71-96ca-21163ad68409'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:39.403583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2706ffc5-e6a2-4e71-96ca-21163ad68409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '308df6c13ef64ce8992f61f31abea8f7', 'request_id': 'req-2706ffc5-e6a2-4e71-96ca-21163ad68409'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:39.411083 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2706ffc5-e6a2-4e71-96ca-21163ad68409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '308df6c13ef64ce8992f61f31abea8f7', 'request_id': 'req-2706ffc5-e6a2-4e71-96ca-21163ad68409'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:39.528594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:39.542713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2706ffc5-e6a2-4e71-96ca-21163ad68409 admin admin] The token cache is being invalidate because project 308df6c13ef64ce8992f61f31abea8f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:39.543261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2706ffc5-e6a2-4e71-96ca-21163ad68409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:39.544457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-2706ffc5-e6a2-4e71-96ca-21163ad68409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:39.546029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 966/1906] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:39 2022] DELETE /identity/v3/projects/308df6c13ef64ce8992f61f31abea8f7 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:39.567800 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef243415-0392-4750-ab59-93a8744eddcf None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:39.568246 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef243415-0392-4750-ab59-93a8744eddcf None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:39.568663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef243415-0392-4750-ab59-93a8744eddcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:39.593186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ef243415-0392-4750-ab59-93a8744eddcf None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:39.594445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ef243415-0392-4750-ab59-93a8744eddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:39.703141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 941/1907] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:39.712903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667bb43-40ba-4d07-91a9-3da938924dc2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:39.713192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667bb43-40ba-4d07-91a9-3da938924dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:39.714850 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667bb43-40ba-4d07-91a9-3da938924dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:39.764270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8667bb43-40ba-4d07-91a9-3da938924dc2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:39.770115 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8667bb43-40ba-4d07-91a9-3da938924dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:39.781807 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 967/1908] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:39.786819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef243415-0392-4750-ab59-93a8744eddcf None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:39.812768 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef243415-0392-4750-ab59-93a8744eddcf None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:39.828078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1079f-d83a-49e9-a27c-10117c3ac684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:39.830072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1079f-d83a-49e9-a27c-10117c3ac684 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:39.830685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1079f-d83a-49e9-a27c-10117c3ac684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:39.831309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1079f-d83a-49e9-a27c-10117c3ac684 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:39.833945 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1079f-d83a-49e9-a27c-10117c3ac684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:39.834885 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1079f-d83a-49e9-a27c-10117c3ac684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:39.843603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1079f-d83a-49e9-a27c-10117c3ac684 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:39.871516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 942/1909] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:49:39.872762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667bb43-40ba-4d07-91a9-3da938924dc2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:39.884045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667bb43-40ba-4d07-91a9-3da938924dc2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:39.889749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82562b32-0dbe-4d15-bd80-916881033488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:39.890488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562b32-0dbe-4d15-bd80-916881033488 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:39.890881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562b32-0dbe-4d15-bd80-916881033488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:39.891246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562b32-0dbe-4d15-bd80-916881033488 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:39.892542 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82562b32-0dbe-4d15-bd80-916881033488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:39.894860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82562b32-0dbe-4d15-bd80-916881033488 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:39.943981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:40.020255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 968/1910] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:39 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:40.025386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1079f-d83a-49e9-a27c-10117c3ac684 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:40.039757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1079f-d83a-49e9-a27c-10117c3ac684 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:40.119733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1aa7d-bd95-41a6-88fb-6d322fd1a751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:40.120837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1aa7d-bd95-41a6-88fb-6d322fd1a751 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:40.121692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1aa7d-bd95-41a6-88fb-6d322fd1a751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:40.122430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1aa7d-bd95-41a6-88fb-6d322fd1a751 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:40.123613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1aa7d-bd95-41a6-88fb-6d322fd1a751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:40.127586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1aa7d-bd95-41a6-88fb-6d322fd1a751 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:40.279787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 943/1911] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:40 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:40.282805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82562b32-0dbe-4d15-bd80-916881033488 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:40.323432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82562b32-0dbe-4d15-bd80-916881033488 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:40.353226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46f10a-22de-4db7-92cb-7aa207983d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:40.354330 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46f10a-22de-4db7-92cb-7aa207983d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:40.354855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46f10a-22de-4db7-92cb-7aa207983d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:40.355534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46f10a-22de-4db7-92cb-7aa207983d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:40.356771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46f10a-22de-4db7-92cb-7aa207983d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:40.359797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46f10a-22de-4db7-92cb-7aa207983d62 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:40.388264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 969/1912] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:40.392389 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1aa7d-bd95-41a6-88fb-6d322fd1a751 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:40.407589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1aa7d-bd95-41a6-88fb-6d322fd1a751 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:40.413163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334ff33-afa4-49d4-8a74-45f2311924bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:40.414538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ff33-afa4-49d4-8a74-45f2311924bf admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:40.414538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ff33-afa4-49d4-8a74-45f2311924bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:40.415199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ff33-afa4-49d4-8a74-45f2311924bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:40.416387 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334ff33-afa4-49d4-8a74-45f2311924bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:40.419485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334ff33-afa4-49d4-8a74-45f2311924bf admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:40.436960 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 944/1913] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:49:40.438368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46f10a-22de-4db7-92cb-7aa207983d62 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:40.449001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46f10a-22de-4db7-92cb-7aa207983d62 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:40.453634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95e271-ff8a-45a9-884b-297f52e63718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:40.454318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95e271-ff8a-45a9-884b-297f52e63718 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:40.454596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95e271-ff8a-45a9-884b-297f52e63718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:40.454871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95e271-ff8a-45a9-884b-297f52e63718 admin admin] PATH_INFO: `/v3/projects/71a4b118f8274580979ae48a297d242b/users/6585e2a014294f6cb02457523a915f5a/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:40.471110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95e271-ff8a-45a9-884b-297f52e63718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a4b118f8274580979ae48a297d242b, user_id=6585e2a014294f6cb02457523a915f5a, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:40.473901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95e271-ff8a-45a9-884b-297f52e63718 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:40.590386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 970/1914] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:40 2022] PUT /identity/v3/projects/71a4b118f8274580979ae48a297d242b/users/6585e2a014294f6cb02457523a915f5a/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:40.603505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d3436-18d6-44cd-b82a-a9a3b19f865f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:40.604008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d3436-18d6-44cd-b82a-a9a3b19f865f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:40.604098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d3436-18d6-44cd-b82a-a9a3b19f865f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:40.632125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-490d3436-18d6-44cd-b82a-a9a3b19f865f None None] MFA Rules not processed for user `6585e2a014294f6cb02457523a915f5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:40.633344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-490d3436-18d6-44cd-b82a-a9a3b19f865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:40.728514 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 945/1915] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:40.733320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95e271-ff8a-45a9-884b-297f52e63718 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:40.805190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95e271-ff8a-45a9-884b-297f52e63718 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:40.811285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c58234-f2de-411f-9590-2d47afe60dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:40.811894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c58234-f2de-411f-9590-2d47afe60dca admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:40.812363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c58234-f2de-411f-9590-2d47afe60dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:40.812779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c58234-f2de-411f-9590-2d47afe60dca admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:40.814348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c58234-f2de-411f-9590-2d47afe60dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:40.817049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c58234-f2de-411f-9590-2d47afe60dca admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:40.944861 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 971/1916] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:40 2022] POST /identity/v3/projects => generated 377 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:40.948919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d3436-18d6-44cd-b82a-a9a3b19f865f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:40.961177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d3436-18d6-44cd-b82a-a9a3b19f865f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:41.032248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a82a56-024e-4c3d-8e65-e96539f71c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:41.032986 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a82a56-024e-4c3d-8e65-e96539f71c90 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:41.033338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a82a56-024e-4c3d-8e65-e96539f71c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:41.033951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a82a56-024e-4c3d-8e65-e96539f71c90 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:41.034882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a82a56-024e-4c3d-8e65-e96539f71c90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:41.036982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a82a56-024e-4c3d-8e65-e96539f71c90 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:41.261890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 946/1917] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:40 2022] POST /identity/v3/users => generated 289 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:41.263034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c58234-f2de-411f-9590-2d47afe60dca admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:41.275149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c58234-f2de-411f-9590-2d47afe60dca admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:41.280696 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef4517-0a4f-4f9c-9af1-6a666f5462a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:41.283135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef4517-0a4f-4f9c-9af1-6a666f5462a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:41.283135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef4517-0a4f-4f9c-9af1-6a666f5462a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:41.283135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef4517-0a4f-4f9c-9af1-6a666f5462a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:41.283135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef4517-0a4f-4f9c-9af1-6a666f5462a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:41.284990 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef4517-0a4f-4f9c-9af1-6a666f5462a8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:41.297494 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 972/1918] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:49:41.302281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a82a56-024e-4c3d-8e65-e96539f71c90 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:41.321270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a82a56-024e-4c3d-8e65-e96539f71c90 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:41.343155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7cf5d-9ff6-4fa7-92b4-19301ad701e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:41.346136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7cf5d-9ff6-4fa7-92b4-19301ad701e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:41.346934 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7cf5d-9ff6-4fa7-92b4-19301ad701e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:41.350801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7cf5d-9ff6-4fa7-92b4-19301ad701e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:41.350801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7cf5d-9ff6-4fa7-92b4-19301ad701e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:41.350801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7cf5d-9ff6-4fa7-92b4-19301ad701e2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:41.365336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 947/1919] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:41.373347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef4517-0a4f-4f9c-9af1-6a666f5462a8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:41.392821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef4517-0a4f-4f9c-9af1-6a666f5462a8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:41.399927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14223fbb-7c91-4404-8b6e-ead2faf753a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:41.402613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14223fbb-7c91-4404-8b6e-ead2faf753a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:41.403232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14223fbb-7c91-4404-8b6e-ead2faf753a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:41.404088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14223fbb-7c91-4404-8b6e-ead2faf753a0 admin admin] PATH_INFO: `/v3/projects/776ecf4dffd94459a8b6a784c417fd2b/users/9e1a52a37ea74e40bcb8560fc191c128/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:41.431253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14223fbb-7c91-4404-8b6e-ead2faf753a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=776ecf4dffd94459a8b6a784c417fd2b, user_id=9e1a52a37ea74e40bcb8560fc191c128, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:41.432765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14223fbb-7c91-4404-8b6e-ead2faf753a0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:41.575250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 973/1920] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:41 2022] PUT /identity/v3/projects/776ecf4dffd94459a8b6a784c417fd2b/users/9e1a52a37ea74e40bcb8560fc191c128/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:41.585290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a402198-61a4-4868-ab7f-9e11a62d3929 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:41.585727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a402198-61a4-4868-ab7f-9e11a62d3929 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:41.586066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a402198-61a4-4868-ab7f-9e11a62d3929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:41.648632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0a402198-61a4-4868-ab7f-9e11a62d3929 None None] MFA Rules not processed for user `9e1a52a37ea74e40bcb8560fc191c128`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:41.661204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0a402198-61a4-4868-ab7f-9e11a62d3929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:41.677511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:41.788466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 948/1921] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:41.792808 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14223fbb-7c91-4404-8b6e-ead2faf753a0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:41.873235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14223fbb-7c91-4404-8b6e-ead2faf753a0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:41.878374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b1e70-3686-4ae5-8445-49b0b3228701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:41.879009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b1e70-3686-4ae5-8445-49b0b3228701 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:41.879263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b1e70-3686-4ae5-8445-49b0b3228701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:41.879533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b1e70-3686-4ae5-8445-49b0b3228701 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:41.880168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b1e70-3686-4ae5-8445-49b0b3228701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:41.882129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b1e70-3686-4ae5-8445-49b0b3228701 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:42.008698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 974/1922] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:41 2022] POST /identity/v3/projects => generated 379 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:42.013066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a402198-61a4-4868-ab7f-9e11a62d3929 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:42.024413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a402198-61a4-4868-ab7f-9e11a62d3929 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:42.093094 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:42.120029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fab7b-e0d5-4783-a0dc-3cf3acec6b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:42.122164 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fab7b-e0d5-4783-a0dc-3cf3acec6b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:42.122164 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fab7b-e0d5-4783-a0dc-3cf3acec6b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:42.122164 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fab7b-e0d5-4783-a0dc-3cf3acec6b74 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:42.123122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fab7b-e0d5-4783-a0dc-3cf3acec6b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:42.124528 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fab7b-e0d5-4783-a0dc-3cf3acec6b74 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:42.270439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 949/1923] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:42 2022] POST /identity/v3/users => generated 296 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:42.274305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b1e70-3686-4ae5-8445-49b0b3228701 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:42.288150 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b1e70-3686-4ae5-8445-49b0b3228701 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:42.296361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535488cf-0cb1-42b3-9bc4-72a480148583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:42.297018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535488cf-0cb1-42b3-9bc4-72a480148583 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:42.297268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535488cf-0cb1-42b3-9bc4-72a480148583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:42.297633 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535488cf-0cb1-42b3-9bc4-72a480148583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:42.298341 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535488cf-0cb1-42b3-9bc4-72a480148583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:42.300245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535488cf-0cb1-42b3-9bc4-72a480148583 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:42.325181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 975/1924] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:42.330599 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fab7b-e0d5-4783-a0dc-3cf3acec6b74 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:42.349348 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fab7b-e0d5-4783-a0dc-3cf3acec6b74 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:42.350225 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03625ba-604a-4409-b479-a31000f2be65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:42.350961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03625ba-604a-4409-b479-a31000f2be65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:42.351240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03625ba-604a-4409-b479-a31000f2be65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:42.351540 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03625ba-604a-4409-b479-a31000f2be65 admin admin] PATH_INFO: `/v3/projects/20c78cf5dd56439f901f04cc2e9fafd0/users/0529ff1af21b456fb8ef122c415399c4/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:42.368939 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03625ba-604a-4409-b479-a31000f2be65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c78cf5dd56439f901f04cc2e9fafd0, user_id=0529ff1af21b456fb8ef122c415399c4, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:42.372380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03625ba-604a-4409-b479-a31000f2be65 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:42.454775 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 950/1925] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:42 2022] PUT /identity/v3/projects/20c78cf5dd56439f901f04cc2e9fafd0/users/0529ff1af21b456fb8ef122c415399c4/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:42.460470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535488cf-0cb1-42b3-9bc4-72a480148583 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:42.471845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535488cf-0cb1-42b3-9bc4-72a480148583 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:42.540765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369dbf1f-cedb-401a-b27b-8f4dd59f8dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:42.541885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369dbf1f-cedb-401a-b27b-8f4dd59f8dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:42.542174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369dbf1f-cedb-401a-b27b-8f4dd59f8dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:42.542438 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369dbf1f-cedb-401a-b27b-8f4dd59f8dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:42.544409 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369dbf1f-cedb-401a-b27b-8f4dd59f8dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:42.548643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369dbf1f-cedb-401a-b27b-8f4dd59f8dc0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:42.565730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 976/1926] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:42.571071 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03625ba-604a-4409-b479-a31000f2be65 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:42.589383 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03625ba-604a-4409-b479-a31000f2be65 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:42.601426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820a444-3df3-41a2-bcc7-dc5ee05e22d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:42.602164 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820a444-3df3-41a2-bcc7-dc5ee05e22d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:42.602451 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820a444-3df3-41a2-bcc7-dc5ee05e22d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:42.602730 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820a444-3df3-41a2-bcc7-dc5ee05e22d0 admin admin] PATH_INFO: `/v3/projects/20c78cf5dd56439f901f04cc2e9fafd0/users/0529ff1af21b456fb8ef122c415399c4/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:42.607621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820a444-3df3-41a2-bcc7-dc5ee05e22d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c78cf5dd56439f901f04cc2e9fafd0, user_id=0529ff1af21b456fb8ef122c415399c4, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:42.610927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820a444-3df3-41a2-bcc7-dc5ee05e22d0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:42.643380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 951/1927] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:42 2022] PUT /identity/v3/projects/20c78cf5dd56439f901f04cc2e9fafd0/users/0529ff1af21b456fb8ef122c415399c4/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:42.670482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fe3e9-02f8-4e01-a48f-fcb8b5fedf2c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:42.670482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fe3e9-02f8-4e01-a48f-fcb8b5fedf2c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:42.670482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fe3e9-02f8-4e01-a48f-fcb8b5fedf2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:42.731411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-dc3fe3e9-02f8-4e01-a48f-fcb8b5fedf2c None None] MFA Rules not processed for user `0529ff1af21b456fb8ef122c415399c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:42.732677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dc3fe3e9-02f8-4e01-a48f-fcb8b5fedf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:42.744206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820a444-3df3-41a2-bcc7-dc5ee05e22d0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:42.747073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f820a444-3df3-41a2-bcc7-dc5ee05e22d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:42.864507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820a444-3df3-41a2-bcc7-dc5ee05e22d0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:42.874334 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:42.875360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:42.875991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:42.876362 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] PATH_INFO: `/v3/users/aa153692a4ea4b228da3bf1ed2853b57` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:42.878057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa153692a4ea4b228da3bf1ed2853b57)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:42.879406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 977/1928] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:42.882855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:42.892699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fe3e9-02f8-4e01-a48f-fcb8b5fedf2c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:42.894241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dc3fe3e9-02f8-4e01-a48f-fcb8b5fedf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:42.973047 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fe3e9-02f8-4e01-a48f-fcb8b5fedf2c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:42.976599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c5757-c63f-4724-a194-7c2523795270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:42.977204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c5757-c63f-4724-a194-7c2523795270 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:42.977486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c5757-c63f-4724-a194-7c2523795270 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:42.977790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c5757-c63f-4724-a194-7c2523795270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:42.988438 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c5757-c63f-4724-a194-7c2523795270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:42.990038 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c5757-c63f-4724-a194-7c2523795270 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:43.000632 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 978/1929] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:43.127049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c5757-c63f-4724-a194-7c2523795270 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:43.130317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ff0c5757-c63f-4724-a194-7c2523795270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:43.169221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa153692a4ea4b228da3bf1ed2853b57'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:43.178412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c5757-c63f-4724-a194-7c2523795270 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:43.191611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df92bf-0b54-45a2-a46d-b0ef7c3d237b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:43.192606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df92bf-0b54-45a2-a46d-b0ef7c3d237b service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:43.212034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df92bf-0b54-45a2-a46d-b0ef7c3d237b service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:43.212034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df92bf-0b54-45a2-a46d-b0ef7c3d237b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:43.271020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df92bf-0b54-45a2-a46d-b0ef7c3d237b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:43.272183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df92bf-0b54-45a2-a46d-b0ef7c3d237b service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:43.278154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa153692a4ea4b228da3bf1ed2853b57'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:43.279059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa153692a4ea4b228da3bf1ed2853b57'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:43.284511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 979/1930] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:43.287112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa153692a4ea4b228da3bf1ed2853b57'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:43.300926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa153692a4ea4b228da3bf1ed2853b57'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:43.308961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 952/1931] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:42 2022] DELETE /identity/v3/users/aa153692a4ea4b228da3bf1ed2853b57 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:43.312356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af774c9a-c2f3-4801-a315-870777251bc9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:43.312651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af774c9a-c2f3-4801-a315-870777251bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:43.312929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af774c9a-c2f3-4801-a315-870777251bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:43.338047 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-af774c9a-c2f3-4801-a315-870777251bc9 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:43.339348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-af774c9a-c2f3-4801-a315-870777251bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:43.404229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 980/1932] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:43.414210 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:43.415631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:43.479467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a262d99-48b6-4afd-9f95-6522a4f242e7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:43.484112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0114f-1c50-4513-9088-90eff9e2e3ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:43.484772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0114f-1c50-4513-9088-90eff9e2e3ab service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:43.485166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0114f-1c50-4513-9088-90eff9e2e3ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:43.485458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0114f-1c50-4513-9088-90eff9e2e3ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:43.496101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0114f-1c50-4513-9088-90eff9e2e3ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:43.497556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0114f-1c50-4513-9088-90eff9e2e3ab service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:43.507390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:43.511169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 953/1933] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:43.775501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af774c9a-c2f3-4801-a315-870777251bc9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:43.777892 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-af774c9a-c2f3-4801-a315-870777251bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:43.796080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af774c9a-c2f3-4801-a315-870777251bc9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:43.804894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e343f-4b61-48fb-ba46-65e8f6ab29d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:43.806564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e343f-4b61-48fb-ba46-65e8f6ab29d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:43.806564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e343f-4b61-48fb-ba46-65e8f6ab29d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:43.808385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e343f-4b61-48fb-ba46-65e8f6ab29d6 admin admin] PATH_INFO: `/v3/users/08bf094c836a4878884a9c735121753f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:43.810080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e343f-4b61-48fb-ba46-65e8f6ab29d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08bf094c836a4878884a9c735121753f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:43.813984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e343f-4b61-48fb-ba46-65e8f6ab29d6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:43.863644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:44.056443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0114f-1c50-4513-9088-90eff9e2e3ab service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:44.057951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dbe0114f-1c50-4513-9088-90eff9e2e3ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:44.080331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0114f-1c50-4513-9088-90eff9e2e3ab service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:44.098933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:44.100004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:44.100423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:44.100806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] PATH_INFO: `/v3/projects/f135df8c000f4a31a4c65e6c2574abcc` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:44.102942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f135df8c000f4a31a4c65e6c2574abcc)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:44.105223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:44.109185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-886e343f-4b61-48fb-ba46-65e8f6ab29d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08bf094c836a4878884a9c735121753f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.256686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-886e343f-4b61-48fb-ba46-65e8f6ab29d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08bf094c836a4878884a9c735121753f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.257415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-886e343f-4b61-48fb-ba46-65e8f6ab29d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08bf094c836a4878884a9c735121753f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.268329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-886e343f-4b61-48fb-ba46-65e8f6ab29d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08bf094c836a4878884a9c735121753f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.282486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-886e343f-4b61-48fb-ba46-65e8f6ab29d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08bf094c836a4878884a9c735121753f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.291476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 981/1934] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:43 2022] DELETE /identity/v3/users/08bf094c836a4878884a9c735121753f => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:44.304234 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1d7b5-3ebf-41aa-a690-b08c28cfb23d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:44.304527 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1d7b5-3ebf-41aa-a690-b08c28cfb23d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:44.304790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1d7b5-3ebf-41aa-a690-b08c28cfb23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:44.340927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f135df8c000f4a31a4c65e6c2574abcc', 'request_id': 'req-df29ef29-5ec2-462a-bdb1-aa9094f247c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.350089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f135df8c000f4a31a4c65e6c2574abcc', 'request_id': 'req-df29ef29-5ec2-462a-bdb1-aa9094f247c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.351972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-aef1d7b5-3ebf-41aa-a690-b08c28cfb23d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:44.362578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-aef1d7b5-3ebf-41aa-a690-b08c28cfb23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:44.375392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f135df8c000f4a31a4c65e6c2574abcc', 'request_id': 'req-df29ef29-5ec2-462a-bdb1-aa9094f247c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.491757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 982/1935] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:44.492789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1d7b5-3ebf-41aa-a690-b08c28cfb23d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:44.494251 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-aef1d7b5-3ebf-41aa-a690-b08c28cfb23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:44.503932 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] The token cache is being invalidate because project f135df8c000f4a31a4c65e6c2574abcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:44.504319 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.505160 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.507011 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 954/1936] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:44 2022] DELETE /identity/v3/projects/f135df8c000f4a31a4c65e6c2574abcc => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:44.513619 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:44.516226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:44.554517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29ef29-5ec2-462a-bdb1-aa9094f247c8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:44.559641 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:44.566209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:44.566209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:44.566209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] PATH_INFO: `/v3/users/106bbf91d2914760a58e7625d8249bdd` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:44.566209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=106bbf91d2914760a58e7625d8249bdd)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:44.566644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:44.595455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1d7b5-3ebf-41aa-a690-b08c28cfb23d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:44.599124 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9616a11-6619-4490-bb10-7177f5ed8be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:44.599742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9616a11-6619-4490-bb10-7177f5ed8be7 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:44.600023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9616a11-6619-4490-bb10-7177f5ed8be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:44.600394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9616a11-6619-4490-bb10-7177f5ed8be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:44.602071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e9616a11-6619-4490-bb10-7177f5ed8be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:44.617144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9616a11-6619-4490-bb10-7177f5ed8be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:44.618641 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9616a11-6619-4490-bb10-7177f5ed8be7 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:44.629956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 983/1937] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:44.668807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:44.818770 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '106bbf91d2914760a58e7625d8249bdd'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.925402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '106bbf91d2914760a58e7625d8249bdd'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.926315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '106bbf91d2914760a58e7625d8249bdd'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.935868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '106bbf91d2914760a58e7625d8249bdd'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.951314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '106bbf91d2914760a58e7625d8249bdd'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:44.958476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 955/1938] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:44 2022] DELETE /identity/v3/users/106bbf91d2914760a58e7625d8249bdd => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:45.173421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9616a11-6619-4490-bb10-7177f5ed8be7 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:45.177761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e9616a11-6619-4490-bb10-7177f5ed8be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:45.217463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9616a11-6619-4490-bb10-7177f5ed8be7 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:45.319658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700de538-d9f8-43ee-84d8-00ac6a55b5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:45.320520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700de538-d9f8-43ee-84d8-00ac6a55b5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:45.320810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700de538-d9f8-43ee-84d8-00ac6a55b5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:45.321189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700de538-d9f8-43ee-84d8-00ac6a55b5ba admin admin] PATH_INFO: `/v3/projects/2786e75ce58040ae96164ac504ce9ec7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:45.323106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700de538-d9f8-43ee-84d8-00ac6a55b5ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=2786e75ce58040ae96164ac504ce9ec7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:45.324546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700de538-d9f8-43ee-84d8-00ac6a55b5ba admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:45.437743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:45.443133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:45.471022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-700de538-d9f8-43ee-84d8-00ac6a55b5ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2786e75ce58040ae96164ac504ce9ec7', 'request_id': 'req-700de538-d9f8-43ee-84d8-00ac6a55b5ba'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:45.471138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dea793-0935-4c16-b164-928f07c740bc admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:45.477433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-700de538-d9f8-43ee-84d8-00ac6a55b5ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2786e75ce58040ae96164ac504ce9ec7', 'request_id': 'req-700de538-d9f8-43ee-84d8-00ac6a55b5ba'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:45.485581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-700de538-d9f8-43ee-84d8-00ac6a55b5ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2786e75ce58040ae96164ac504ce9ec7', 'request_id': 'req-700de538-d9f8-43ee-84d8-00ac6a55b5ba'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:45.504569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:45.562918 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:45.563639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:45.564046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:45.564264 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] PATH_INFO: `/v3/users/6c6a63d12bce4014898d450819f10ef3` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:45.565727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c6a63d12bce4014898d450819f10ef3)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:45.568579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:45.608976 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-700de538-d9f8-43ee-84d8-00ac6a55b5ba admin admin] The token cache is being invalidate because project 2786e75ce58040ae96164ac504ce9ec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:45.609366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-700de538-d9f8-43ee-84d8-00ac6a55b5ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:45.610808 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-700de538-d9f8-43ee-84d8-00ac6a55b5ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:45.617413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 984/1939] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:45 2022] DELETE /identity/v3/projects/2786e75ce58040ae96164ac504ce9ec7 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:45.619381 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba55397-a109-4857-b509-be3409410800 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:45.619662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba55397-a109-4857-b509-be3409410800 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:45.619943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba55397-a109-4857-b509-be3409410800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:45.659818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-fba55397-a109-4857-b509-be3409410800 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:45.661128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fba55397-a109-4857-b509-be3409410800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:45.674311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 985/1940] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:45.683468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bd8f1-b943-4fe5-991a-7ff2ed25cbd9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:45.683767 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bd8f1-b943-4fe5-991a-7ff2ed25cbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:45.684048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bd8f1-b943-4fe5-991a-7ff2ed25cbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:45.711917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d49bd8f1-b943-4fe5-991a-7ff2ed25cbd9 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:45.713967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d49bd8f1-b943-4fe5-991a-7ff2ed25cbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:45.729508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 986/1941] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:45.730574 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bd8f1-b943-4fe5-991a-7ff2ed25cbd9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:45.743202 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:45.748196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bd8f1-b943-4fe5-991a-7ff2ed25cbd9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:45.768644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127e195-fb90-431f-8f4b-7706de567c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:45.769574 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127e195-fb90-431f-8f4b-7706de567c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:45.770321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127e195-fb90-431f-8f4b-7706de567c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:45.770830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127e195-fb90-431f-8f4b-7706de567c2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:45.772699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127e195-fb90-431f-8f4b-7706de567c2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:45.773346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127e195-fb90-431f-8f4b-7706de567c2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:45.775253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127e195-fb90-431f-8f4b-7706de567c2d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:45.799742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 987/1942] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:45.804154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127e195-fb90-431f-8f4b-7706de567c2d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:45.817261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127e195-fb90-431f-8f4b-7706de567c2d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:45.823377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6ff8a-4810-44cb-8c03-9e400e19e979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:45.824002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6ff8a-4810-44cb-8c03-9e400e19e979 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:45.824287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6ff8a-4810-44cb-8c03-9e400e19e979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:45.825043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6ff8a-4810-44cb-8c03-9e400e19e979 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:45.825740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6ff8a-4810-44cb-8c03-9e400e19e979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:45.827716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6ff8a-4810-44cb-8c03-9e400e19e979 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:45.864957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c6a63d12bce4014898d450819f10ef3'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:45.972143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c6a63d12bce4014898d450819f10ef3'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:45.973006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c6a63d12bce4014898d450819f10ef3'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:45.981740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 988/1943] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:45 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:45.984013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c6a63d12bce4014898d450819f10ef3'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:45.987954 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6ff8a-4810-44cb-8c03-9e400e19e979 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:45.990130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-30e6ff8a-4810-44cb-8c03-9e400e19e979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:45.998171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c6a63d12bce4014898d450819f10ef3'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:46.018973 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 956/1944] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:45 2022] DELETE /identity/v3/users/6c6a63d12bce4014898d450819f10ef3 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:46.018973 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:46.018973 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:46.081496 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6ff8a-4810-44cb-8c03-9e400e19e979 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:46.100526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962840b-f763-4c11-b794-1bb2f8b6a145 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:46.106189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:46.107629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:46.107629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:46.107629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] PATH_INFO: `/v3/projects/4fa69aa874ef46ef9359383bdf4e4c37` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:46.108749 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa69aa874ef46ef9359383bdf4e4c37)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:46.112685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:46.172305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e720227-e4cd-414a-8368-70d51ed697f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:46.172929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e720227-e4cd-414a-8368-70d51ed697f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:46.173215 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e720227-e4cd-414a-8368-70d51ed697f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:46.173977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e720227-e4cd-414a-8368-70d51ed697f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:46.175596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e720227-e4cd-414a-8368-70d51ed697f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:46.178104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e720227-e4cd-414a-8368-70d51ed697f7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:46.548538 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 989/1945] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:45 2022] POST /identity/v3/users => generated 289 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:46.548538 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab65c3-fecc-45a5-8abe-95823db53128 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:46.548538 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab65c3-fecc-45a5-8abe-95823db53128 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:46.548538 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab65c3-fecc-45a5-8abe-95823db53128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:46.550120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa69aa874ef46ef9359383bdf4e4c37', 'request_id': 'req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:46.550120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa69aa874ef46ef9359383bdf4e4c37', 'request_id': 'req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:46.550120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa69aa874ef46ef9359383bdf4e4c37', 'request_id': 'req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:46.550883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4cab65c3-fecc-45a5-8abe-95823db53128 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:46.550883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4cab65c3-fecc-45a5-8abe-95823db53128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:46.550883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 990/1946] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:46.550883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab65c3-fecc-45a5-8abe-95823db53128 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:46.550883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab65c3-fecc-45a5-8abe-95823db53128 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:46.551254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998205e-42af-4f62-bc2d-3036fbb1437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:46.551254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998205e-42af-4f62-bc2d-3036fbb1437a admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:46.551254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998205e-42af-4f62-bc2d-3036fbb1437a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:46.551254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998205e-42af-4f62-bc2d-3036fbb1437a admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:46.551582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998205e-42af-4f62-bc2d-3036fbb1437a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:46.551582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998205e-42af-4f62-bc2d-3036fbb1437a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:46.551582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 991/1947] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:46.551582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998205e-42af-4f62-bc2d-3036fbb1437a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:46.551582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5998205e-42af-4f62-bc2d-3036fbb1437a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:46.551914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] The token cache is being invalidate because project 4fa69aa874ef46ef9359383bdf4e4c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:46.551914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:46.551914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:46.551914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 957/1948] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:46 2022] DELETE /identity/v3/projects/4fa69aa874ef46ef9359383bdf4e4c37 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:46.551914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:46.552300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:46.552390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998205e-42af-4f62-bc2d-3036fbb1437a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:46.552390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aae68-c9c7-41fe-97bd-cafab5dd6249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:46.552390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aae68-c9c7-41fe-97bd-cafab5dd6249 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:46.552390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aae68-c9c7-41fe-97bd-cafab5dd6249 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:46.552903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aae68-c9c7-41fe-97bd-cafab5dd6249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:46.552903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ff3aae68-c9c7-41fe-97bd-cafab5dd6249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:46.554182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e9fc5-3573-4fb8-ac8c-ed8b73ed5106 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:46.559501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa65091-8ab3-44dc-b172-88fa17c79fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:46.560481 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa65091-8ab3-44dc-b172-88fa17c79fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:46.561111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa65091-8ab3-44dc-b172-88fa17c79fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:46.561639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa65091-8ab3-44dc-b172-88fa17c79fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:46.562631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa65091-8ab3-44dc-b172-88fa17c79fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:46.565274 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa65091-8ab3-44dc-b172-88fa17c79fe2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:46.566607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aae68-c9c7-41fe-97bd-cafab5dd6249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:46.568260 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aae68-c9c7-41fe-97bd-cafab5dd6249 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:46.581905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 992/1949] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:46.583571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a46d4-b21c-451d-8216-f3c9393eba3e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:46.583913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a46d4-b21c-451d-8216-f3c9393eba3e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:46.584191 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a46d4-b21c-451d-8216-f3c9393eba3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:46.586080 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 958/1950] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:46.598821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa65091-8ab3-44dc-b172-88fa17c79fe2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:46.611895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa65091-8ab3-44dc-b172-88fa17c79fe2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:46.620813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2611cc2-eaa1-4565-9f23-ddfcb156481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:46.622257 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611cc2-eaa1-4565-9f23-ddfcb156481c admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:46.623800 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611cc2-eaa1-4565-9f23-ddfcb156481c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:46.624494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611cc2-eaa1-4565-9f23-ddfcb156481c admin admin] PATH_INFO: `/v3/projects/1a321d3be51341e6bc68b315a2781610/users/2ed45d354d5a4b7ca8ea6839307e44a7/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:46.643088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-282a46d4-b21c-451d-8216-f3c9393eba3e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:46.647288 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-282a46d4-b21c-451d-8216-f3c9393eba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:46.663264 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2611cc2-eaa1-4565-9f23-ddfcb156481c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a321d3be51341e6bc68b315a2781610, user_id=2ed45d354d5a4b7ca8ea6839307e44a7, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:46.668940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 993/1951] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:46.672976 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2611cc2-eaa1-4565-9f23-ddfcb156481c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:46.678971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04af98-0683-45b8-8201-41aa0aca8d70 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:46.679318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04af98-0683-45b8-8201-41aa0aca8d70 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:46.679523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04af98-0683-45b8-8201-41aa0aca8d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:46.707695 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4d04af98-0683-45b8-8201-41aa0aca8d70 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:46.709227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4d04af98-0683-45b8-8201-41aa0aca8d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:46.764555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 994/1952] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:46.764555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04af98-0683-45b8-8201-41aa0aca8d70 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:46.764555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04af98-0683-45b8-8201-41aa0aca8d70 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:46.764555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3a97b-aa02-4881-ac8c-50fdb56da13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:46.765562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3a97b-aa02-4881-ac8c-50fdb56da13d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:46.765562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3a97b-aa02-4881-ac8c-50fdb56da13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:46.765562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3a97b-aa02-4881-ac8c-50fdb56da13d admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:46.765562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3a97b-aa02-4881-ac8c-50fdb56da13d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:46.765562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3a97b-aa02-4881-ac8c-50fdb56da13d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:46.765562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3a97b-aa02-4881-ac8c-50fdb56da13d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:46.765982 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 995/1953] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:49:46.765982 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3a97b-aa02-4881-ac8c-50fdb56da13d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:46.775490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3a97b-aa02-4881-ac8c-50fdb56da13d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:46.782111 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ce00a-084a-411e-81f0-dc7d248da242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:46.782111 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ce00a-084a-411e-81f0-dc7d248da242 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:46.782111 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ce00a-084a-411e-81f0-dc7d248da242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:46.782111 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ce00a-084a-411e-81f0-dc7d248da242 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:46.782507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ce00a-084a-411e-81f0-dc7d248da242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:46.784137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ce00a-084a-411e-81f0-dc7d248da242 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:46.811105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 959/1954] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:46 2022] PUT /identity/v3/projects/1a321d3be51341e6bc68b315a2781610/users/2ed45d354d5a4b7ca8ea6839307e44a7/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:46.817858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54e484-fcff-4e95-9ee9-401e89cfde06 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:46.818172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54e484-fcff-4e95-9ee9-401e89cfde06 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:46.818450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54e484-fcff-4e95-9ee9-401e89cfde06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:46.854472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5c54e484-fcff-4e95-9ee9-401e89cfde06 None None] MFA Rules not processed for user `2ed45d354d5a4b7ca8ea6839307e44a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:46.855830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5c54e484-fcff-4e95-9ee9-401e89cfde06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:46.904699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 996/1955] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:46 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:46.908839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ce00a-084a-411e-81f0-dc7d248da242 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:46.933882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 960/1956] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:46.942425 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54e484-fcff-4e95-9ee9-401e89cfde06 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:47.016090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ce00a-084a-411e-81f0-dc7d248da242 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:47.023605 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfbb9b-bef6-4103-a0d4-ff3adfa27344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:47.025193 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfbb9b-bef6-4103-a0d4-ff3adfa27344 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:47.025667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfbb9b-bef6-4103-a0d4-ff3adfa27344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:47.027700 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfbb9b-bef6-4103-a0d4-ff3adfa27344 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:47.028864 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfbb9b-bef6-4103-a0d4-ff3adfa27344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:47.031512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfbb9b-bef6-4103-a0d4-ff3adfa27344 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:47.035188 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54e484-fcff-4e95-9ee9-401e89cfde06 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:47.041085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a66cac-c884-4d18-bf56-8eb1e41519e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:47.041836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a66cac-c884-4d18-bf56-8eb1e41519e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:47.042092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a66cac-c884-4d18-bf56-8eb1e41519e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:47.044943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a66cac-c884-4d18-bf56-8eb1e41519e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:47.044943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a66cac-c884-4d18-bf56-8eb1e41519e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:47.045781 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a66cac-c884-4d18-bf56-8eb1e41519e8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:47.282792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 997/1957] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:46 2022] POST /identity/v3/users => generated 290 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:47.283919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfbb9b-bef6-4103-a0d4-ff3adfa27344 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:47.285013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 961/1958] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:46 2022] POST /identity/v3/projects => generated 377 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:47.285972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a66cac-c884-4d18-bf56-8eb1e41519e8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:47.295441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfbb9b-bef6-4103-a0d4-ff3adfa27344 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:47.361913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a66cac-c884-4d18-bf56-8eb1e41519e8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:47.365064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb6d45-0d12-4f6e-b951-9e6459d68019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:47.365848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb6d45-0d12-4f6e-b951-9e6459d68019 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:47.366303 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb6d45-0d12-4f6e-b951-9e6459d68019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:47.366683 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb6d45-0d12-4f6e-b951-9e6459d68019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:47.367886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb6d45-0d12-4f6e-b951-9e6459d68019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:47.378960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb6d45-0d12-4f6e-b951-9e6459d68019 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:47.379076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75f59f-85ca-47ec-91df-b67a869731b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:47.379076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75f59f-85ca-47ec-91df-b67a869731b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:47.379076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75f59f-85ca-47ec-91df-b67a869731b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:47.379076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75f59f-85ca-47ec-91df-b67a869731b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:47.379431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75f59f-85ca-47ec-91df-b67a869731b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:47.379431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75f59f-85ca-47ec-91df-b67a869731b6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:47.384121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 998/1959] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:47.389253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb6d45-0d12-4f6e-b951-9e6459d68019 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:47.401133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb6d45-0d12-4f6e-b951-9e6459d68019 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:47.407153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e9c54-352b-4484-8432-b4b92b7e84f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:47.407844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e9c54-352b-4484-8432-b4b92b7e84f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:47.408175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e9c54-352b-4484-8432-b4b92b7e84f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:47.408508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e9c54-352b-4484-8432-b4b92b7e84f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:47.409359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e9c54-352b-4484-8432-b4b92b7e84f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:47.412048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e9c54-352b-4484-8432-b4b92b7e84f1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:47.433352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 999/1960] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:49:47.434896 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e9c54-352b-4484-8432-b4b92b7e84f1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:47.438801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ab9e9c54-352b-4484-8432-b4b92b7e84f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:47.454806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e9c54-352b-4484-8432-b4b92b7e84f1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:47.462210 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901693f9-f375-47ed-821a-c9b077786298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:47.462965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901693f9-f375-47ed-821a-c9b077786298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:47.464380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901693f9-f375-47ed-821a-c9b077786298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:47.464380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901693f9-f375-47ed-821a-c9b077786298 admin admin] PATH_INFO: `/v3/projects/b86f28e15c8f42f4bc89e28fea05cead` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:47.465457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901693f9-f375-47ed-821a-c9b077786298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b86f28e15c8f42f4bc89e28fea05cead)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:47.467584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901693f9-f375-47ed-821a-c9b077786298 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:47.491478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 962/1961] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:47 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:47.493365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75f59f-85ca-47ec-91df-b67a869731b6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:47.511439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75f59f-85ca-47ec-91df-b67a869731b6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:47.516505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795edc36-a7c2-48fe-bfa8-1be67979f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:47.517198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795edc36-a7c2-48fe-bfa8-1be67979f1d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:47.517706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795edc36-a7c2-48fe-bfa8-1be67979f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:47.518036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795edc36-a7c2-48fe-bfa8-1be67979f1d8 admin admin] PATH_INFO: `/v3/projects/12aca9385ca4477ea1678c2450cde39a/users/d6acd659008f4bfe8e3489ef72a0f041/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:47.535628 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795edc36-a7c2-48fe-bfa8-1be67979f1d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12aca9385ca4477ea1678c2450cde39a, user_id=d6acd659008f4bfe8e3489ef72a0f041, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:47.538367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795edc36-a7c2-48fe-bfa8-1be67979f1d8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:47.543300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:47.681232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 963/1962] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:47 2022] PUT /identity/v3/projects/12aca9385ca4477ea1678c2450cde39a/users/d6acd659008f4bfe8e3489ef72a0f041/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:47.682038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795edc36-a7c2-48fe-bfa8-1be67979f1d8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:47.719473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-901693f9-f375-47ed-821a-c9b077786298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b86f28e15c8f42f4bc89e28fea05cead', 'request_id': 'req-901693f9-f375-47ed-821a-c9b077786298'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:47.731594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-901693f9-f375-47ed-821a-c9b077786298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b86f28e15c8f42f4bc89e28fea05cead', 'request_id': 'req-901693f9-f375-47ed-821a-c9b077786298'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:47.741450 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-901693f9-f375-47ed-821a-c9b077786298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b86f28e15c8f42f4bc89e28fea05cead', 'request_id': 'req-901693f9-f375-47ed-821a-c9b077786298'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:47.800758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:47.806384 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795edc36-a7c2-48fe-bfa8-1be67979f1d8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:47.845659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-901693f9-f375-47ed-821a-c9b077786298 admin admin] The token cache is being invalidate because project b86f28e15c8f42f4bc89e28fea05cead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:47.851369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-901693f9-f375-47ed-821a-c9b077786298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:47.851369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-901693f9-f375-47ed-821a-c9b077786298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:47.851369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1000/1963] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:47 2022] DELETE /identity/v3/projects/b86f28e15c8f42f4bc89e28fea05cead => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:47.852649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02b36-0911-4036-b086-112f88a1dd46 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:47.854122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02b36-0911-4036-b086-112f88a1dd46 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:47.855781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02b36-0911-4036-b086-112f88a1dd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:47.914863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4da02b36-0911-4036-b086-112f88a1dd46 None None] MFA Rules not processed for user `d6acd659008f4bfe8e3489ef72a0f041`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:47.916131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4da02b36-0911-4036-b086-112f88a1dd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:47.940211 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf92926-9125-4b5c-a0f1-b699f977d291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:47.940846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf92926-9125-4b5c-a0f1-b699f977d291 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:47.941115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf92926-9125-4b5c-a0f1-b699f977d291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:47.941615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf92926-9125-4b5c-a0f1-b699f977d291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:47.942436 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf92926-9125-4b5c-a0f1-b699f977d291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:47.944653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf92926-9125-4b5c-a0f1-b699f977d291 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:47.966077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 964/1964] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:47.967368 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf92926-9125-4b5c-a0f1-b699f977d291 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:47.970407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3cf92926-9125-4b5c-a0f1-b699f977d291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:47.987757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf92926-9125-4b5c-a0f1-b699f977d291 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:47.994620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:47.994620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:47.994856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:47.995037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] PATH_INFO: `/v3/users/27c215f0741d49c28f9c0c5b08d2c1cf` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:47.996210 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c215f0741d49c28f9c0c5b08d2c1cf)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:47.998743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:48.040740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1001/1965] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:48.042037 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02b36-0911-4036-b086-112f88a1dd46 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:48.043598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4da02b36-0911-4036-b086-112f88a1dd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:48.062118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02b36-0911-4036-b086-112f88a1dd46 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:48.067744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ad853-99c8-48a7-b143-9c496b92598f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:48.068519 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ad853-99c8-48a7-b143-9c496b92598f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:48.068807 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ad853-99c8-48a7-b143-9c496b92598f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:48.069082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ad853-99c8-48a7-b143-9c496b92598f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:48.070448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ad853-99c8-48a7-b143-9c496b92598f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:48.072238 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ad853-99c8-48a7-b143-9c496b92598f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:48.089236 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1002/1966] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:48.089236 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ad853-99c8-48a7-b143-9c496b92598f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:48.089236 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-014ad853-99c8-48a7-b143-9c496b92598f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:48.103051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ad853-99c8-48a7-b143-9c496b92598f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:48.108023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03b93e-0a80-401f-b9b9-8510c52fce8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:48.108649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03b93e-0a80-401f-b9b9-8510c52fce8a admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:48.108925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03b93e-0a80-401f-b9b9-8510c52fce8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:48.109200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03b93e-0a80-401f-b9b9-8510c52fce8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:48.110441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03b93e-0a80-401f-b9b9-8510c52fce8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:48.112375 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03b93e-0a80-401f-b9b9-8510c52fce8a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:48.253272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1003/1967] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:48 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:48.254326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03b93e-0a80-401f-b9b9-8510c52fce8a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:48.282650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c215f0741d49c28f9c0c5b08d2c1cf'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:48.382285 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c215f0741d49c28f9c0c5b08d2c1cf'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:48.383076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c215f0741d49c28f9c0c5b08d2c1cf'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:48.383199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03b93e-0a80-401f-b9b9-8510c52fce8a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:48.390681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d3377-71f7-4e13-a701-274b47cb957a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:48.392775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d3377-71f7-4e13-a701-274b47cb957a admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:48.393193 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d3377-71f7-4e13-a701-274b47cb957a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:48.393485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d3377-71f7-4e13-a701-274b47cb957a admin admin] PATH_INFO: `/v3/projects/a02d743fcb964cbebff55822a729ea55/users/0a11abcc18764fe08a745e96122d5d4e/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:48.398858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c215f0741d49c28f9c0c5b08d2c1cf'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:48.420015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d3377-71f7-4e13-a701-274b47cb957a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a02d743fcb964cbebff55822a729ea55, user_id=0a11abcc18764fe08a745e96122d5d4e, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:48.425402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d3377-71f7-4e13-a701-274b47cb957a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:48.433682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c215f0741d49c28f9c0c5b08d2c1cf'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:48.442095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 965/1968] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:47 2022] DELETE /identity/v3/users/27c215f0741d49c28f9c0c5b08d2c1cf => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:48.442959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:48.444591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:48.521539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1004/1969] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:48 2022] PUT /identity/v3/projects/a02d743fcb964cbebff55822a729ea55/users/0a11abcc18764fe08a745e96122d5d4e/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:48.522691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d3377-71f7-4e13-a701-274b47cb957a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:48.534150 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-064d3377-71f7-4e13-a701-274b47cb957a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:48.548062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd92332-f23a-4fba-92fd-f4613684e3ad admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:48.627562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d3377-71f7-4e13-a701-274b47cb957a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:48.629627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5449c9-c287-4aa7-9fcb-deaa8a5c82f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:48.630491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5449c9-c287-4aa7-9fcb-deaa8a5c82f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:48.630930 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5449c9-c287-4aa7-9fcb-deaa8a5c82f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:48.631423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5449c9-c287-4aa7-9fcb-deaa8a5c82f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:48.632302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5449c9-c287-4aa7-9fcb-deaa8a5c82f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:48.633275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:48.634112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:48.634471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:48.634819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] PATH_INFO: `/v3/users/6585e2a014294f6cb02457523a915f5a` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:48.635505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5449c9-c287-4aa7-9fcb-deaa8a5c82f5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:48.636317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6585e2a014294f6cb02457523a915f5a)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:48.643929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:48.764447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 966/1970] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:48 2022] POST /identity/v3/users => generated 296 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:48.764447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add922b0-83ed-411c-a232-3e66199c46b3 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:48.764447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add922b0-83ed-411c-a232-3e66199c46b3 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:48.764447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add922b0-83ed-411c-a232-3e66199c46b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:48.801386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-add922b0-83ed-411c-a232-3e66199c46b3 None None] MFA Rules not processed for user `0a11abcc18764fe08a745e96122d5d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:48.802774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-add922b0-83ed-411c-a232-3e66199c46b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:48.881013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6585e2a014294f6cb02457523a915f5a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:48.934332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 967/1971] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:48.934332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add922b0-83ed-411c-a232-3e66199c46b3 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:48.945036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add922b0-83ed-411c-a232-3e66199c46b3 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:48.950985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4a915-6b1e-4e57-a162-5dcfb0e69a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:48.951615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4a915-6b1e-4e57-a162-5dcfb0e69a4e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:48.951926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4a915-6b1e-4e57-a162-5dcfb0e69a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:48.952273 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4a915-6b1e-4e57-a162-5dcfb0e69a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:48.953237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4a915-6b1e-4e57-a162-5dcfb0e69a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:48.954525 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6585e2a014294f6cb02457523a915f5a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:48.954979 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4a915-6b1e-4e57-a162-5dcfb0e69a4e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:48.955824 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6585e2a014294f6cb02457523a915f5a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:48.967301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 968/1972] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:48.967802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4a915-6b1e-4e57-a162-5dcfb0e69a4e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:48.970141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-49e4a915-6b1e-4e57-a162-5dcfb0e69a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:48.972992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6585e2a014294f6cb02457523a915f5a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:48.994388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4a915-6b1e-4e57-a162-5dcfb0e69a4e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:49.002084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5d03a-9ed7-4103-85b6-89db05dcc3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:49.002822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5d03a-9ed7-4103-85b6-89db05dcc3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.003221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5d03a-9ed7-4103-85b6-89db05dcc3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.003648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5d03a-9ed7-4103-85b6-89db05dcc3c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.004491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5d03a-9ed7-4103-85b6-89db05dcc3c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:49.005613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6585e2a014294f6cb02457523a915f5a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:49.007648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5d03a-9ed7-4103-85b6-89db05dcc3c8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:49.017249 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1005/1973] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:48 2022] DELETE /identity/v3/users/6585e2a014294f6cb02457523a915f5a => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:49.021628 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:49.021628 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:49.105565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5f913-4aa1-46d6-90f9-39d74ebe095b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:49.111735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ed6c0-d4b3-4d25-b91d-5438ee9419fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:49.113244 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ed6c0-d4b3-4d25-b91d-5438ee9419fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.113549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ed6c0-d4b3-4d25-b91d-5438ee9419fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.118840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ed6c0-d4b3-4d25-b91d-5438ee9419fb admin admin] PATH_INFO: `/v3/projects/0c696a5b7fd349a98befcc33fe6cf277/users/b9c9dc18da82478b8892e76e4d1ed783/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.131455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ed6c0-d4b3-4d25-b91d-5438ee9419fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c696a5b7fd349a98befcc33fe6cf277, user_id=b9c9dc18da82478b8892e76e4d1ed783, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:49.134337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ed6c0-d4b3-4d25-b91d-5438ee9419fb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:49.145164 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 969/1974] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:48 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:49.147055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c2356-be12-48ae-be88-c32a7fed2bbb None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.147848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c2356-be12-48ae-be88-c32a7fed2bbb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.147848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c2356-be12-48ae-be88-c32a7fed2bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.165863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:49.168424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-188c2356-be12-48ae-be88-c32a7fed2bbb None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:49.169765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-188c2356-be12-48ae-be88-c32a7fed2bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:49.223566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1006/1975] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:49 2022] PUT /identity/v3/projects/0c696a5b7fd349a98befcc33fe6cf277/users/b9c9dc18da82478b8892e76e4d1ed783/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:49.224322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ed6c0-d4b3-4d25-b91d-5438ee9419fb admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:49.226152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d89ed6c0-d4b3-4d25-b91d-5438ee9419fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:49.240166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 970/1976] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:49.241012 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c2356-be12-48ae-be88-c32a7fed2bbb None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:49.255173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c2356-be12-48ae-be88-c32a7fed2bbb None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:49.297139 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ed6c0-d4b3-4d25-b91d-5438ee9419fb admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:49.302684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:49.304401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.304401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.304401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] PATH_INFO: `/v3/projects/618d186d70a744c1bf55f7a764d04623` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.305321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=618d186d70a744c1bf55f7a764d04623)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:49.307292 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:49.314619 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b343e3-4859-420f-8ad7-08187ba692a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:49.315297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b343e3-4859-420f-8ad7-08187ba692a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.315771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b343e3-4859-420f-8ad7-08187ba692a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.315974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b343e3-4859-420f-8ad7-08187ba692a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.316610 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b343e3-4859-420f-8ad7-08187ba692a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:49.318586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b343e3-4859-420f-8ad7-08187ba692a3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:49.436751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 971/1977] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:49 2022] POST /identity/v3/users => generated 295 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:49.436751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b343e3-4859-420f-8ad7-08187ba692a3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:49.444329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:49.448121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '618d186d70a744c1bf55f7a764d04623', 'request_id': 'req-c9ab7abf-4ded-49fe-8682-c34bb84bf508'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:49.448806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b343e3-4859-420f-8ad7-08187ba692a3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:49.453957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '618d186d70a744c1bf55f7a764d04623', 'request_id': 'req-c9ab7abf-4ded-49fe-8682-c34bb84bf508'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:49.460273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '618d186d70a744c1bf55f7a764d04623', 'request_id': 'req-c9ab7abf-4ded-49fe-8682-c34bb84bf508'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:49.512220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518976be-005e-4273-b103-2c4b8f8ba945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:49.512739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518976be-005e-4273-b103-2c4b8f8ba945 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.513021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518976be-005e-4273-b103-2c4b8f8ba945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.513297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518976be-005e-4273-b103-2c4b8f8ba945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.513989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518976be-005e-4273-b103-2c4b8f8ba945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:49.515583 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518976be-005e-4273-b103-2c4b8f8ba945 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:49.525595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 972/1978] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:49.526513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518976be-005e-4273-b103-2c4b8f8ba945 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:49.528251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-518976be-005e-4273-b103-2c4b8f8ba945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:49.574162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] The token cache is being invalidate because project 618d186d70a744c1bf55f7a764d04623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:49.574734 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:49.575655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:49.577412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1007/1979] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:49 2022] DELETE /identity/v3/projects/618d186d70a744c1bf55f7a764d04623 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:49.578608 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:49.580786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:49.595945 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518976be-005e-4273-b103-2c4b8f8ba945 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:49.598904 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab7abf-4ded-49fe-8682-c34bb84bf508 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:49.602511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fa62f-2cf0-45a7-aa32-0d7ea1b9464c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:49.602511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fa62f-2cf0-45a7-aa32-0d7ea1b9464c service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.602511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fa62f-2cf0-45a7-aa32-0d7ea1b9464c service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.602511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fa62f-2cf0-45a7-aa32-0d7ea1b9464c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.603003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-450fa62f-2cf0-45a7-aa32-0d7ea1b9464c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:49.604372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e763381-39c3-43ca-9748-f9807ee7364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:49.604975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e763381-39c3-43ca-9748-f9807ee7364d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.605255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e763381-39c3-43ca-9748-f9807ee7364d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.605544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e763381-39c3-43ca-9748-f9807ee7364d admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.606222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e763381-39c3-43ca-9748-f9807ee7364d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:49.607916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e763381-39c3-43ca-9748-f9807ee7364d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:49.616409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fa62f-2cf0-45a7-aa32-0d7ea1b9464c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:49.617989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fa62f-2cf0-45a7-aa32-0d7ea1b9464c service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:49.619447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1008/1980] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:49.623723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e763381-39c3-43ca-9748-f9807ee7364d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:49.625499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4e763381-39c3-43ca-9748-f9807ee7364d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:49.633593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 973/1981] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:49.635293 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3892fd1-ce18-413e-93b2-a1f7729806b0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.635581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3892fd1-ce18-413e-93b2-a1f7729806b0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.635857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3892fd1-ce18-413e-93b2-a1f7729806b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.638702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e763381-39c3-43ca-9748-f9807ee7364d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:49.647617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6008a2-bccd-4f0b-a179-164ab09588f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:49.648615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6008a2-bccd-4f0b-a179-164ab09588f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.648899 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6008a2-bccd-4f0b-a179-164ab09588f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.649178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6008a2-bccd-4f0b-a179-164ab09588f1 admin admin] PATH_INFO: `/v3/projects/0c696a5b7fd349a98befcc33fe6cf277/users/b9c9dc18da82478b8892e76e4d1ed783/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.651917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6008a2-bccd-4f0b-a179-164ab09588f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c696a5b7fd349a98befcc33fe6cf277, user_id=b9c9dc18da82478b8892e76e4d1ed783, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:49.654403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6008a2-bccd-4f0b-a179-164ab09588f1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:49.657994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b3892fd1-ce18-413e-93b2-a1f7729806b0 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:49.659173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b3892fd1-ce18-413e-93b2-a1f7729806b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:49.666695 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1009/1982] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:49 2022] PUT /identity/v3/projects/0c696a5b7fd349a98befcc33fe6cf277/users/b9c9dc18da82478b8892e76e4d1ed783/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:49.667656 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6008a2-bccd-4f0b-a179-164ab09588f1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:49.739306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 974/1983] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:49.739774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6008a2-bccd-4f0b-a179-164ab09588f1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:49.740449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cbaa8-5185-4843-99eb-b6a90274265b None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.740772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cbaa8-5185-4843-99eb-b6a90274265b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.741190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cbaa8-5185-4843-99eb-b6a90274265b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.746673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea2fe5-6eff-4c04-a3b6-2e4a05cc0c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:49.747341 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea2fe5-6eff-4c04-a3b6-2e4a05cc0c9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.747622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea2fe5-6eff-4c04-a3b6-2e4a05cc0c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.747896 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea2fe5-6eff-4c04-a3b6-2e4a05cc0c9b admin admin] PATH_INFO: `/v3/projects/6a8460f6cb7c451db702992d62de1b1e/users/185b33480dbb4a099de834c8dad310f3/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.766675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea2fe5-6eff-4c04-a3b6-2e4a05cc0c9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8460f6cb7c451db702992d62de1b1e, user_id=185b33480dbb4a099de834c8dad310f3, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:49.769355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea2fe5-6eff-4c04-a3b6-2e4a05cc0c9b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:49.771458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-2e1cbaa8-5185-4843-99eb-b6a90274265b None None] MFA Rules not processed for user `b9c9dc18da82478b8892e76e4d1ed783`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:49.772764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2e1cbaa8-5185-4843-99eb-b6a90274265b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:49.853868 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1010/1984] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:49 2022] PUT /identity/v3/projects/6a8460f6cb7c451db702992d62de1b1e/users/185b33480dbb4a099de834c8dad310f3/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:49.855709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78180b-3126-4493-a36b-aa577dabd6f7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.856169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78180b-3126-4493-a36b-aa577dabd6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.857507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78180b-3126-4493-a36b-aa577dabd6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.860971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 975/1985] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:49.862585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cbaa8-5185-4843-99eb-b6a90274265b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:49.880530 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ab78180b-3126-4493-a36b-aa577dabd6f7 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:49.882017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ab78180b-3126-4493-a36b-aa577dabd6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:49.882154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cbaa8-5185-4843-99eb-b6a90274265b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:49.964660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1011/1986] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:49.964660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78180b-3126-4493-a36b-aa577dabd6f7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:49.964660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ab78180b-3126-4493-a36b-aa577dabd6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:49.967551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dead09-ed90-4390-805b-8aba3517be05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:49.968172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dead09-ed90-4390-805b-8aba3517be05 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:49.968449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dead09-ed90-4390-805b-8aba3517be05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:49.968717 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dead09-ed90-4390-805b-8aba3517be05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:49.970153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dead09-ed90-4390-805b-8aba3517be05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:49.971786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dead09-ed90-4390-805b-8aba3517be05 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:49.987855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 976/1987] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:49.988721 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dead09-ed90-4390-805b-8aba3517be05 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:49.999119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dead09-ed90-4390-805b-8aba3517be05 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:50.004945 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7a8f3-4aec-43c0-8f60-4f0875ceda5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:50.005120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:50.005574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7a8f3-4aec-43c0-8f60-4f0875ceda5b admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:50.005989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7a8f3-4aec-43c0-8f60-4f0875ceda5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:50.006397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7a8f3-4aec-43c0-8f60-4f0875ceda5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:50.007059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7a8f3-4aec-43c0-8f60-4f0875ceda5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:50.007419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7a8f3-4aec-43c0-8f60-4f0875ceda5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:50.009105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7a8f3-4aec-43c0-8f60-4f0875ceda5b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:50.024257 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 977/1988] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:50.024922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7a8f3-4aec-43c0-8f60-4f0875ceda5b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:50.041575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7a8f3-4aec-43c0-8f60-4f0875ceda5b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:50.042910 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78180b-3126-4493-a36b-aa577dabd6f7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:50.047296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f875981-2e16-4bb2-9f00-74c84d1dcb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:50.047574 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e8bb0-d1e5-4937-9755-3531951797ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:50.048016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f875981-2e16-4bb2-9f00-74c84d1dcb7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:50.048274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e8bb0-d1e5-4937-9755-3531951797ff service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:50.048471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f875981-2e16-4bb2-9f00-74c84d1dcb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:50.048552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e8bb0-d1e5-4937-9755-3531951797ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:50.048719 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f875981-2e16-4bb2-9f00-74c84d1dcb7f admin admin] PATH_INFO: `/v3/projects/6a8460f6cb7c451db702992d62de1b1e/users/185b33480dbb4a099de834c8dad310f3/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:50.048905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e8bb0-d1e5-4937-9755-3531951797ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:50.051015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f875981-2e16-4bb2-9f00-74c84d1dcb7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8460f6cb7c451db702992d62de1b1e, user_id=185b33480dbb4a099de834c8dad310f3, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:50.055267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f875981-2e16-4bb2-9f00-74c84d1dcb7f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:50.065454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 978/1989] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:50 2022] PUT /identity/v3/projects/6a8460f6cb7c451db702992d62de1b1e/users/185b33480dbb4a099de834c8dad310f3/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:50.066356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f875981-2e16-4bb2-9f00-74c84d1dcb7f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:50.112705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e8bb0-d1e5-4937-9755-3531951797ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:50.114196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e8bb0-d1e5-4937-9755-3531951797ff service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:50.123866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1012/1990] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:50.125080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7048d-9ac4-46af-b28a-ba3e12bb3f3b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:50.125928 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7048d-9ac4-46af-b28a-ba3e12bb3f3b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:50.126198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7048d-9ac4-46af-b28a-ba3e12bb3f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:50.129993 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f875981-2e16-4bb2-9f00-74c84d1dcb7f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:50.133085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:50.138090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e3e25-e0ae-4a29-9ae1-e11642f28a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:50.138874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e3e25-e0ae-4a29-9ae1-e11642f28a28 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:50.139067 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e3e25-e0ae-4a29-9ae1-e11642f28a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:50.139717 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e3e25-e0ae-4a29-9ae1-e11642f28a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:50.140394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e3e25-e0ae-4a29-9ae1-e11642f28a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:50.142903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e3e25-e0ae-4a29-9ae1-e11642f28a28 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:50.158674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c8b7048d-9ac4-46af-b28a-ba3e12bb3f3b None None] MFA Rules not processed for user `185b33480dbb4a099de834c8dad310f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:50.161836 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c8b7048d-9ac4-46af-b28a-ba3e12bb3f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:50.238398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1013/1991] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:50.239556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7048d-9ac4-46af-b28a-ba3e12bb3f3b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:50.240942 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c8b7048d-9ac4-46af-b28a-ba3e12bb3f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:50.259179 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7048d-9ac4-46af-b28a-ba3e12bb3f3b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:50.273406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:50.274156 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:50.274534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:50.274915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] PATH_INFO: `/v3/projects/71a4b118f8274580979ae48a297d242b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:50.276294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a4b118f8274580979ae48a297d242b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:50.279445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:50.291581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 979/1992] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:50 2022] POST /identity/v3/projects => generated 381 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:50.291581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e3e25-e0ae-4a29-9ae1-e11642f28a28 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:50.294303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-310e3e25-e0ae-4a29-9ae1-e11642f28a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:50.369381 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e3e25-e0ae-4a29-9ae1-e11642f28a28 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:50.373420 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b4ec5-15a0-451b-8152-72f954d8ad08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:50.378294 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b4ec5-15a0-451b-8152-72f954d8ad08 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:50.378581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b4ec5-15a0-451b-8152-72f954d8ad08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:50.378851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b4ec5-15a0-451b-8152-72f954d8ad08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:50.383957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1b8b4ec5-15a0-451b-8152-72f954d8ad08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:50.421592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a4b118f8274580979ae48a297d242b', 'request_id': 'req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:50.429077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a4b118f8274580979ae48a297d242b', 'request_id': 'req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:50.439006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a4b118f8274580979ae48a297d242b', 'request_id': 'req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:50.460416 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b4ec5-15a0-451b-8152-72f954d8ad08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:50.462170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b4ec5-15a0-451b-8152-72f954d8ad08 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:50.472365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 980/1993] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:50.473837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b4ec5-15a0-451b-8152-72f954d8ad08 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:50.550432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] The token cache is being invalidate because project 71a4b118f8274580979ae48a297d242b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:50.550901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:50.552479 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:50.557925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1014/1994] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:50 2022] DELETE /identity/v3/projects/71a4b118f8274580979ae48a297d242b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:50.559004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:50.560769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:50.567661 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b4ec5-15a0-451b-8152-72f954d8ad08 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:50.610345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:50.644074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c12639-96b1-450a-8309-405a5c686982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:50.645021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c12639-96b1-450a-8309-405a5c686982 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:50.645150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c12639-96b1-450a-8309-405a5c686982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:50.645571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c12639-96b1-450a-8309-405a5c686982 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:50.646284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c12639-96b1-450a-8309-405a5c686982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:50.648457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c12639-96b1-450a-8309-405a5c686982 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:50.668018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bdbe0-8ca6-49b4-a8cf-af2bf5ff67bf admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:50.678112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:50.678112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:50.678112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:50.678112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] PATH_INFO: `/v3/users/9e1a52a37ea74e40bcb8560fc191c128` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:50.678418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e1a52a37ea74e40bcb8560fc191c128)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:50.678418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:50.771683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 981/1995] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:50 2022] POST /identity/v3/users => generated 291 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:50.775629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c12639-96b1-450a-8309-405a5c686982 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:50.777339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-20c12639-96b1-450a-8309-405a5c686982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:50.789421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c12639-96b1-450a-8309-405a5c686982 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:50.794027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d6c2c-b44a-4fce-bc5b-7d816269fa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:50.794611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d6c2c-b44a-4fce-bc5b-7d816269fa13 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:50.794897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d6c2c-b44a-4fce-bc5b-7d816269fa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:50.795170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d6c2c-b44a-4fce-bc5b-7d816269fa13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:50.795803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d6c2c-b44a-4fce-bc5b-7d816269fa13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:50.798041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d6c2c-b44a-4fce-bc5b-7d816269fa13 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:50.815289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 982/1996] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:49:50.816271 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d6c2c-b44a-4fce-bc5b-7d816269fa13 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:50.826498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d6c2c-b44a-4fce-bc5b-7d816269fa13 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:50.831005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4288723-a5f0-428c-a0a6-c58f9aabd885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:50.831586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4288723-a5f0-428c-a0a6-c58f9aabd885 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:50.831864 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4288723-a5f0-428c-a0a6-c58f9aabd885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:50.832136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4288723-a5f0-428c-a0a6-c58f9aabd885 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:50.832765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4288723-a5f0-428c-a0a6-c58f9aabd885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:50.834362 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4288723-a5f0-428c-a0a6-c58f9aabd885 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:50.845056 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 983/1997] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:49:50.846016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4288723-a5f0-428c-a0a6-c58f9aabd885 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:50.847567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e4288723-a5f0-428c-a0a6-c58f9aabd885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:50.860650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4288723-a5f0-428c-a0a6-c58f9aabd885 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:50.865416 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b248a1e-7611-498d-b2f2-17faea75f7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:50.866055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b248a1e-7611-498d-b2f2-17faea75f7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:50.866340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b248a1e-7611-498d-b2f2-17faea75f7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:50.866614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b248a1e-7611-498d-b2f2-17faea75f7e9 admin admin] PATH_INFO: `/v3/users/d6acd659008f4bfe8e3489ef72a0f041` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:50.867745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b248a1e-7611-498d-b2f2-17faea75f7e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6acd659008f4bfe8e3489ef72a0f041)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:50.869663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b248a1e-7611-498d-b2f2-17faea75f7e9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:50.927215 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e1a52a37ea74e40bcb8560fc191c128'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:51.039271 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e1a52a37ea74e40bcb8560fc191c128'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:51.039700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:51.040301 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e1a52a37ea74e40bcb8560fc191c128'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:51.049323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e1a52a37ea74e40bcb8560fc191c128'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:51.062184 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e1a52a37ea74e40bcb8560fc191c128'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:51.070705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1015/1998] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:50 2022] DELETE /identity/v3/users/9e1a52a37ea74e40bcb8560fc191c128 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:51.071564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:51.073921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:51.088088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2b248a1e-7611-498d-b2f2-17faea75f7e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6acd659008f4bfe8e3489ef72a0f041'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:51.147189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61f854-a350-457e-b45f-1cf643f2fbd7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:51.152323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9ae7c-151b-4e30-ab19-d2e9aed16b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:51.153021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9ae7c-151b-4e30-ab19-d2e9aed16b36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:51.153308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9ae7c-151b-4e30-ab19-d2e9aed16b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:51.153613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9ae7c-151b-4e30-ab19-d2e9aed16b36 admin admin] PATH_INFO: `/v3/projects/3358298e9a2c426d86d0b22068b83146/users/56a8c5b85a4845008533fff3a8ab5675/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:51.168551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9ae7c-151b-4e30-ab19-d2e9aed16b36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3358298e9a2c426d86d0b22068b83146, user_id=56a8c5b85a4845008533fff3a8ab5675, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:51.174221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9ae7c-151b-4e30-ab19-d2e9aed16b36 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:51.199295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2b248a1e-7611-498d-b2f2-17faea75f7e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6acd659008f4bfe8e3489ef72a0f041'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:51.199938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2b248a1e-7611-498d-b2f2-17faea75f7e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6acd659008f4bfe8e3489ef72a0f041'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:51.208514 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2b248a1e-7611-498d-b2f2-17faea75f7e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6acd659008f4bfe8e3489ef72a0f041'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:51.222336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2b248a1e-7611-498d-b2f2-17faea75f7e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6acd659008f4bfe8e3489ef72a0f041'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:51.228872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 984/1999] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:50 2022] DELETE /identity/v3/users/d6acd659008f4bfe8e3489ef72a0f041 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:51.233938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d2bd-a05a-4c54-ab09-2b08e4e45f8a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:51.234234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d2bd-a05a-4c54-ab09-2b08e4e45f8a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:51.234510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d2bd-a05a-4c54-ab09-2b08e4e45f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:51.256184 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ff81d2bd-a05a-4c54-ab09-2b08e4e45f8a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:51.257409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ff81d2bd-a05a-4c54-ab09-2b08e4e45f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:51.274826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1016/2000] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:51 2022] PUT /identity/v3/projects/3358298e9a2c426d86d0b22068b83146/users/56a8c5b85a4845008533fff3a8ab5675/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:51.285814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fd6ae-1563-4ba3-87ad-ab0edfaa5497 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:51.286093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fd6ae-1563-4ba3-87ad-ab0edfaa5497 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:51.286821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fd6ae-1563-4ba3-87ad-ab0edfaa5497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:51.292715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:51.324132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-112fd6ae-1563-4ba3-87ad-ab0edfaa5497 None None] MFA Rules not processed for user `56a8c5b85a4845008533fff3a8ab5675`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:51.325581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-112fd6ae-1563-4ba3-87ad-ab0edfaa5497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:51.327339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 985/2001] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:51.347027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81d2bd-a05a-4c54-ab09-2b08e4e45f8a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:51.348622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ff81d2bd-a05a-4c54-ab09-2b08e4e45f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:51.462072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1017/2002] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:51.466087 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112fd6ae-1563-4ba3-87ad-ab0edfaa5497 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:51.467608 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-112fd6ae-1563-4ba3-87ad-ab0edfaa5497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:51.477076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81d2bd-a05a-4c54-ab09-2b08e4e45f8a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:51.539647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a25389-ab8e-48a0-9212-7bacd0568a47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:51.540495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a25389-ab8e-48a0-9212-7bacd0568a47 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:51.540850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a25389-ab8e-48a0-9212-7bacd0568a47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:51.541198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a25389-ab8e-48a0-9212-7bacd0568a47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:51.562282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112fd6ae-1563-4ba3-87ad-ab0edfaa5497 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:51.567531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639eee56-0cdf-48c2-a9b9-936bd4671127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:51.568148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639eee56-0cdf-48c2-a9b9-936bd4671127 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:51.568434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639eee56-0cdf-48c2-a9b9-936bd4671127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:51.568712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639eee56-0cdf-48c2-a9b9-936bd4671127 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:51.569370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639eee56-0cdf-48c2-a9b9-936bd4671127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:51.571388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639eee56-0cdf-48c2-a9b9-936bd4671127 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:51.625855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a25389-ab8e-48a0-9212-7bacd0568a47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:51.627474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a25389-ab8e-48a0-9212-7bacd0568a47 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:51.644108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 986/2003] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:51.673099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1018/2004] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:51 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:51.674332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a25389-ab8e-48a0-9212-7bacd0568a47 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:51.685480 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a25389-ab8e-48a0-9212-7bacd0568a47 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:51.747094 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74183058-6f3e-4446-8346-2d70f0ec28c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:51.747716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74183058-6f3e-4446-8346-2d70f0ec28c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:51.747993 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74183058-6f3e-4446-8346-2d70f0ec28c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:51.748262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74183058-6f3e-4446-8346-2d70f0ec28c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:51.748893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74183058-6f3e-4446-8346-2d70f0ec28c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:51.750936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74183058-6f3e-4446-8346-2d70f0ec28c3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:51.769171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639eee56-0cdf-48c2-a9b9-936bd4671127 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:51.771302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-639eee56-0cdf-48c2-a9b9-936bd4671127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:51.785706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639eee56-0cdf-48c2-a9b9-936bd4671127 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:51.791344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:51.803125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:51.804870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:51.805636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] PATH_INFO: `/v3/projects/776ecf4dffd94459a8b6a784c417fd2b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:51.808457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] RBAC: Authorizing `identity:delete_project(project_id=776ecf4dffd94459a8b6a784c417fd2b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:51.811338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:51.905596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 987/2005] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:51 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:51.907729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74183058-6f3e-4446-8346-2d70f0ec28c3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:51.917882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74183058-6f3e-4446-8346-2d70f0ec28c3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:51.942780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '776ecf4dffd94459a8b6a784c417fd2b', 'request_id': 'req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:51.955388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '776ecf4dffd94459a8b6a784c417fd2b', 'request_id': 'req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:51.965172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '776ecf4dffd94459a8b6a784c417fd2b', 'request_id': 'req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.000205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:52.006235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee64e20-1103-4888-8565-6bda1247bad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:52.007419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee64e20-1103-4888-8565-6bda1247bad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:52.011394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee64e20-1103-4888-8565-6bda1247bad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:52.011394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee64e20-1103-4888-8565-6bda1247bad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:52.011394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee64e20-1103-4888-8565-6bda1247bad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:52.011394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee64e20-1103-4888-8565-6bda1247bad7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:52.021371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 988/2006] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:49:52.025629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee64e20-1103-4888-8565-6bda1247bad7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:52.036141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee64e20-1103-4888-8565-6bda1247bad7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:52.040980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100f63e-ebc9-477a-b008-1be393b80e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:52.043735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100f63e-ebc9-477a-b008-1be393b80e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:52.043735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100f63e-ebc9-477a-b008-1be393b80e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:52.043735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100f63e-ebc9-477a-b008-1be393b80e6d admin admin] PATH_INFO: `/v3/projects/7de36d10b6d445dba3259fda83ecbe68/users/e51e92dde3994b019f485da92f595788/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:52.059321 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100f63e-ebc9-477a-b008-1be393b80e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7de36d10b6d445dba3259fda83ecbe68, user_id=e51e92dde3994b019f485da92f595788, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:52.062021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100f63e-ebc9-477a-b008-1be393b80e6d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:52.073033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] The token cache is being invalidate because project 776ecf4dffd94459a8b6a784c417fd2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:52.073415 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.074189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.075856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1019/2007] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:51 2022] DELETE /identity/v3/projects/776ecf4dffd94459a8b6a784c417fd2b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:52.080759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:52.082212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:52.095494 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f99ac4-db76-4df3-bc8b-f0cdbcc5909b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:52.100119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e802f-33c3-40d2-9b0c-4b611867e7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:52.100743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e802f-33c3-40d2-9b0c-4b611867e7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:52.101020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e802f-33c3-40d2-9b0c-4b611867e7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:52.101850 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e802f-33c3-40d2-9b0c-4b611867e7b7 admin admin] PATH_INFO: `/v3/users/0529ff1af21b456fb8ef122c415399c4` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:52.102474 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e802f-33c3-40d2-9b0c-4b611867e7b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0529ff1af21b456fb8ef122c415399c4)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:52.104401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e802f-33c3-40d2-9b0c-4b611867e7b7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:52.139060 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 989/2008] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:52 2022] PUT /identity/v3/projects/7de36d10b6d445dba3259fda83ecbe68/users/e51e92dde3994b019f485da92f595788/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:52.143423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100f63e-ebc9-477a-b008-1be393b80e6d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:52.145033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6100f63e-ebc9-477a-b008-1be393b80e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:52.210925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100f63e-ebc9-477a-b008-1be393b80e6d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:52.215764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf77b17-2993-4dd1-9e44-7f4a6e8d7a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:52.216400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf77b17-2993-4dd1-9e44-7f4a6e8d7a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:52.216682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf77b17-2993-4dd1-9e44-7f4a6e8d7a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:52.216955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf77b17-2993-4dd1-9e44-7f4a6e8d7a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:52.217634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf77b17-2993-4dd1-9e44-7f4a6e8d7a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:52.219844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf77b17-2993-4dd1-9e44-7f4a6e8d7a70 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:52.229999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 990/2009] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:49:52.234443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf77b17-2993-4dd1-9e44-7f4a6e8d7a70 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:52.245350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf77b17-2993-4dd1-9e44-7f4a6e8d7a70 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:52.251453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c2ad0-1390-4100-8588-77d5bd37e5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:52.252118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c2ad0-1390-4100-8588-77d5bd37e5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:52.252396 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c2ad0-1390-4100-8588-77d5bd37e5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:52.252674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c2ad0-1390-4100-8588-77d5bd37e5a4 admin admin] PATH_INFO: `/v3/projects/7de36d10b6d445dba3259fda83ecbe68/users/e51e92dde3994b019f485da92f595788/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:52.254670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c2ad0-1390-4100-8588-77d5bd37e5a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7de36d10b6d445dba3259fda83ecbe68, user_id=e51e92dde3994b019f485da92f595788, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:52.257995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c2ad0-1390-4100-8588-77d5bd37e5a4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:52.275292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 991/2010] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:52 2022] PUT /identity/v3/projects/7de36d10b6d445dba3259fda83ecbe68/users/e51e92dde3994b019f485da92f595788/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:52.285720 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c2ad0-1390-4100-8588-77d5bd37e5a4 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:52.287406 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-601c2ad0-1390-4100-8588-77d5bd37e5a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:52.340005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-387e802f-33c3-40d2-9b0c-4b611867e7b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0529ff1af21b456fb8ef122c415399c4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.356502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c2ad0-1390-4100-8588-77d5bd37e5a4 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:52.361127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:52.361785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:52.362064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:52.362336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] PATH_INFO: `/v3/projects/12aca9385ca4477ea1678c2450cde39a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:52.363457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=12aca9385ca4477ea1678c2450cde39a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:52.365352 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:52.428231 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-387e802f-33c3-40d2-9b0c-4b611867e7b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0529ff1af21b456fb8ef122c415399c4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.429206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-387e802f-33c3-40d2-9b0c-4b611867e7b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0529ff1af21b456fb8ef122c415399c4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.431856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:52.439158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-387e802f-33c3-40d2-9b0c-4b611867e7b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0529ff1af21b456fb8ef122c415399c4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.452063 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-387e802f-33c3-40d2-9b0c-4b611867e7b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0529ff1af21b456fb8ef122c415399c4'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.460106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1020/2011] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:52 2022] DELETE /identity/v3/users/0529ff1af21b456fb8ef122c415399c4 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:52.466556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d812e2f-f672-4c5c-b554-a9b2a1b04a0e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:52.466556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d812e2f-f672-4c5c-b554-a9b2a1b04a0e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:52.466556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d812e2f-f672-4c5c-b554-a9b2a1b04a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:52.495546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0d812e2f-f672-4c5c-b554-a9b2a1b04a0e None None] MFA Rules not processed for user `e51e92dde3994b019f485da92f595788`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:52.497028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0d812e2f-f672-4c5c-b554-a9b2a1b04a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:52.591785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12aca9385ca4477ea1678c2450cde39a', 'request_id': 'req-5287b6d6-5283-45f8-87a6-44714041a0ba'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.591785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12aca9385ca4477ea1678c2450cde39a', 'request_id': 'req-5287b6d6-5283-45f8-87a6-44714041a0ba'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.592069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1021/2012] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:52.592882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12aca9385ca4477ea1678c2450cde39a', 'request_id': 'req-5287b6d6-5283-45f8-87a6-44714041a0ba'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.609846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d812e2f-f672-4c5c-b554-a9b2a1b04a0e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:52.609846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0d812e2f-f672-4c5c-b554-a9b2a1b04a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:52.664084 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d812e2f-f672-4c5c-b554-a9b2a1b04a0e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:52.725989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] The token cache is being invalidate because project 12aca9385ca4477ea1678c2450cde39a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:52.726425 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.727235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:52.732141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 992/2013] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:52 2022] DELETE /identity/v3/projects/12aca9385ca4477ea1678c2450cde39a => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:52.734149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:52.736304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6dd59-4d87-47f0-a299-d09dfe83b906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:52.736939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6dd59-4d87-47f0-a299-d09dfe83b906 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:52.737220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6dd59-4d87-47f0-a299-d09dfe83b906 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:52.737495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6dd59-4d87-47f0-a299-d09dfe83b906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:52.739634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:52.745040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f8a6dd59-4d87-47f0-a299-d09dfe83b906 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:52.858710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6dd59-4d87-47f0-a299-d09dfe83b906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:52.860637 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6dd59-4d87-47f0-a299-d09dfe83b906 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:52.869421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287b6d6-5283-45f8-87a6-44714041a0ba admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:52.873178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1022/2014] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:52.882274 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483f6ac-4eb4-4eea-a905-f21d23c854d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:52.883201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483f6ac-4eb4-4eea-a905-f21d23c854d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:52.883839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483f6ac-4eb4-4eea-a905-f21d23c854d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:52.884301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483f6ac-4eb4-4eea-a905-f21d23c854d5 admin admin] PATH_INFO: `/v3/users/b9c9dc18da82478b8892e76e4d1ed783` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:52.885908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483f6ac-4eb4-4eea-a905-f21d23c854d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c9dc18da82478b8892e76e4d1ed783)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:52.888412 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483f6ac-4eb4-4eea-a905-f21d23c854d5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:52.956959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6dd59-4d87-47f0-a299-d09dfe83b906 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:52.958686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f8a6dd59-4d87-47f0-a299-d09dfe83b906 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:52.979442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6dd59-4d87-47f0-a299-d09dfe83b906 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:52.984248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627e9e2-cefc-4afa-aa6c-30e49bd52cb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:52.984869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627e9e2-cefc-4afa-aa6c-30e49bd52cb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:52.985147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627e9e2-cefc-4afa-aa6c-30e49bd52cb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:52.985421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627e9e2-cefc-4afa-aa6c-30e49bd52cb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:52.987606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e627e9e2-cefc-4afa-aa6c-30e49bd52cb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:53.022450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:53.048567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627e9e2-cefc-4afa-aa6c-30e49bd52cb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:53.050418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627e9e2-cefc-4afa-aa6c-30e49bd52cb4 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:53.059487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1023/2015] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:53.104674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627e9e2-cefc-4afa-aa6c-30e49bd52cb4 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:53.106454 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e627e9e2-cefc-4afa-aa6c-30e49bd52cb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:53.132606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627e9e2-cefc-4afa-aa6c-30e49bd52cb4 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:53.150085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be6de9-342b-4af0-85ec-adab1712e84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:53.150854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be6de9-342b-4af0-85ec-adab1712e84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:53.152009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be6de9-342b-4af0-85ec-adab1712e84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:53.152291 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be6de9-342b-4af0-85ec-adab1712e84f admin admin] PATH_INFO: `/v3/projects/20c78cf5dd56439f901f04cc2e9fafd0` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:53.153658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be6de9-342b-4af0-85ec-adab1712e84f admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c78cf5dd56439f901f04cc2e9fafd0)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:53.155707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be6de9-342b-4af0-85ec-adab1712e84f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:53.207181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6483f6ac-4eb4-4eea-a905-f21d23c854d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c9dc18da82478b8892e76e4d1ed783'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:53.303151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-99be6de9-342b-4af0-85ec-adab1712e84f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c78cf5dd56439f901f04cc2e9fafd0', 'request_id': 'req-99be6de9-342b-4af0-85ec-adab1712e84f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:53.308597 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-99be6de9-342b-4af0-85ec-adab1712e84f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c78cf5dd56439f901f04cc2e9fafd0', 'request_id': 'req-99be6de9-342b-4af0-85ec-adab1712e84f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:53.315166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-99be6de9-342b-4af0-85ec-adab1712e84f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c78cf5dd56439f901f04cc2e9fafd0', 'request_id': 'req-99be6de9-342b-4af0-85ec-adab1712e84f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:53.324926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6483f6ac-4eb4-4eea-a905-f21d23c854d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c9dc18da82478b8892e76e4d1ed783'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:53.325688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6483f6ac-4eb4-4eea-a905-f21d23c854d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c9dc18da82478b8892e76e4d1ed783'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:53.333807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6483f6ac-4eb4-4eea-a905-f21d23c854d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c9dc18da82478b8892e76e4d1ed783'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:53.345752 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6483f6ac-4eb4-4eea-a905-f21d23c854d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c9dc18da82478b8892e76e4d1ed783'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:53.351731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 993/2016] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:52 2022] DELETE /identity/v3/users/b9c9dc18da82478b8892e76e4d1ed783 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:53.418905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-99be6de9-342b-4af0-85ec-adab1712e84f admin admin] The token cache is being invalidate because project 20c78cf5dd56439f901f04cc2e9fafd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:53.419254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-99be6de9-342b-4af0-85ec-adab1712e84f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:53.420222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-99be6de9-342b-4af0-85ec-adab1712e84f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:53.421921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1024/2017] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:53 2022] DELETE /identity/v3/projects/20c78cf5dd56439f901f04cc2e9fafd0 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:53.428844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6456e1-c7f1-4177-a0a1-e8d2c91fb6e1 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:53.429180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6456e1-c7f1-4177-a0a1-e8d2c91fb6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:53.429478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6456e1-c7f1-4177-a0a1-e8d2c91fb6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:53.449868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-0b6456e1-c7f1-4177-a0a1-e8d2c91fb6e1 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:53.452029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0b6456e1-c7f1-4177-a0a1-e8d2c91fb6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:53.511570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 994/2018] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:53.519854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47700a06-b2dc-4c1e-b6f5-9880d1639e1c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:53.520139 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47700a06-b2dc-4c1e-b6f5-9880d1639e1c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:53.520410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47700a06-b2dc-4c1e-b6f5-9880d1639e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:53.543938 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-47700a06-b2dc-4c1e-b6f5-9880d1639e1c None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:53.545218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-47700a06-b2dc-4c1e-b6f5-9880d1639e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:53.561308 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6456e1-c7f1-4177-a0a1-e8d2c91fb6e1 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:53.562432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1025/2019] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:53.573071 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6456e1-c7f1-4177-a0a1-e8d2c91fb6e1 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:53.578580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f375a-95ff-4de4-96eb-a2ebc4c81adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:53.579390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f375a-95ff-4de4-96eb-a2ebc4c81adb admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:53.579653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f375a-95ff-4de4-96eb-a2ebc4c81adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:53.579926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f375a-95ff-4de4-96eb-a2ebc4c81adb admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:53.580620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f375a-95ff-4de4-96eb-a2ebc4c81adb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:53.580959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f375a-95ff-4de4-96eb-a2ebc4c81adb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:53.582593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f375a-95ff-4de4-96eb-a2ebc4c81adb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:53.596262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 995/2020] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:53.600827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47700a06-b2dc-4c1e-b6f5-9880d1639e1c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:53.611322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47700a06-b2dc-4c1e-b6f5-9880d1639e1c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:53.616295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc88b6f-4315-4b11-ba5a-d7a362f206c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:53.616901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc88b6f-4315-4b11-ba5a-d7a362f206c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:53.617252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc88b6f-4315-4b11-ba5a-d7a362f206c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:53.617559 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc88b6f-4315-4b11-ba5a-d7a362f206c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:53.618181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc88b6f-4315-4b11-ba5a-d7a362f206c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:53.620165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc88b6f-4315-4b11-ba5a-d7a362f206c8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:53.769351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1026/2021] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:53 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:53.770497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f375a-95ff-4de4-96eb-a2ebc4c81adb admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:53.780748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f375a-95ff-4de4-96eb-a2ebc4c81adb admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:53.808442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc88b6f-4315-4b11-ba5a-d7a362f206c8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:53.810396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cdc88b6f-4315-4b11-ba5a-d7a362f206c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:53.851231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e46a6-524b-4c17-9fd6-ec81ef7fab67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:53.851231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e46a6-524b-4c17-9fd6-ec81ef7fab67 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:53.851231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e46a6-524b-4c17-9fd6-ec81ef7fab67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:53.851231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e46a6-524b-4c17-9fd6-ec81ef7fab67 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:53.851530 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e46a6-524b-4c17-9fd6-ec81ef7fab67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:53.852249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e46a6-524b-4c17-9fd6-ec81ef7fab67 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:53.878820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc88b6f-4315-4b11-ba5a-d7a362f206c8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:53.883556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:53.884184 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:53.884464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:53.884767 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] PATH_INFO: `/v3/projects/0c696a5b7fd349a98befcc33fe6cf277` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:53.885968 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c696a5b7fd349a98befcc33fe6cf277)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:53.887939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:53.959864 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 996/2022] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:53 2022] POST /identity/v3/users => generated 284 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:49:53.972168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e46a6-524b-4c17-9fd6-ec81ef7fab67 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:53.990599 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e46a6-524b-4c17-9fd6-ec81ef7fab67 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:53.996955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a44a3-5112-4657-bfd1-07df74e045fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:53.998454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a44a3-5112-4657-bfd1-07df74e045fd admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:53.998777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a44a3-5112-4657-bfd1-07df74e045fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:53.999498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a44a3-5112-4657-bfd1-07df74e045fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.000502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a44a3-5112-4657-bfd1-07df74e045fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:54.002603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a44a3-5112-4657-bfd1-07df74e045fd admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:54.039810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 997/2023] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:54.041145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c696a5b7fd349a98befcc33fe6cf277', 'request_id': 'req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:54.050508 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a44a3-5112-4657-bfd1-07df74e045fd admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:54.058435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c696a5b7fd349a98befcc33fe6cf277', 'request_id': 'req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:54.068307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c696a5b7fd349a98befcc33fe6cf277', 'request_id': 'req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:54.069977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a44a3-5112-4657-bfd1-07df74e045fd admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:54.190909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] The token cache is being invalidate because project 0c696a5b7fd349a98befcc33fe6cf277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:54.191691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:54.192153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:54.193777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1027/2024] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:53 2022] DELETE /identity/v3/projects/0c696a5b7fd349a98befcc33fe6cf277 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:54.197722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:54.209952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:54.270640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:54.271612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d31cd-2840-4aa2-91b8-0a636d1633d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:54.272831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d31cd-2840-4aa2-91b8-0a636d1633d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.273495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d31cd-2840-4aa2-91b8-0a636d1633d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.274614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d31cd-2840-4aa2-91b8-0a636d1633d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.275919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d31cd-2840-4aa2-91b8-0a636d1633d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:54.278829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d31cd-2840-4aa2-91b8-0a636d1633d6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:54.303045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 998/2025] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:54.305901 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0581c-e350-4ef0-a0fc-2ce031d60352 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.306519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0581c-e350-4ef0-a0fc-2ce031d60352 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.306989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0581c-e350-4ef0-a0fc-2ce031d60352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.322166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5b4a8-5b13-4548-98bb-2d5975f2dd70 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:54.323310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:54.327898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ec35d-5567-4368-b0a7-cce0334c6e70 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e51e92dde3994b019f485da92f595788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de36d10b6d445dba3259fda83ecbe68', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:54.329084 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ec35d-5567-4368-b0a7-cce0334c6e70 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.329364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ec35d-5567-4368-b0a7-cce0334c6e70 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.329669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ec35d-5567-4368-b0a7-cce0334c6e70 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.330321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ec35d-5567-4368-b0a7-cce0334c6e70 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:54.330624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ec35d-5567-4368-b0a7-cce0334c6e70 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:54.332339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ec35d-5567-4368-b0a7-cce0334c6e70 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:54.336412 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-f8b0581c-e350-4ef0-a0fc-2ce031d60352 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:54.337682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f8b0581c-e350-4ef0-a0fc-2ce031d60352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:54.349653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1028/2026] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:54.349653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ec35d-5567-4368-b0a7-cce0334c6e70 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:54.351409 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-872ec35d-5567-4368-b0a7-cce0334c6e70 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:54.353956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 999/2027] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:54.354885 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0581c-e350-4ef0-a0fc-2ce031d60352 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:54.365650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0581c-e350-4ef0-a0fc-2ce031d60352 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:54.366358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ec35d-5567-4368-b0a7-cce0334c6e70 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:54.371161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73d088-4c3e-4af5-9cf6-2b017082f8f7 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e51e92dde3994b019f485da92f595788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de36d10b6d445dba3259fda83ecbe68', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:54.371896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73d088-4c3e-4af5-9cf6-2b017082f8f7 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.372189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73d088-4c3e-4af5-9cf6-2b017082f8f7 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.372293 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f051a7e-36a9-4e93-8344-720c53bc62f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:54.372595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73d088-4c3e-4af5-9cf6-2b017082f8f7 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.372858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f051a7e-36a9-4e93-8344-720c53bc62f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.373176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73d088-4c3e-4af5-9cf6-2b017082f8f7 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:54.373294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f051a7e-36a9-4e93-8344-720c53bc62f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.373448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f051a7e-36a9-4e93-8344-720c53bc62f5 admin admin] PATH_INFO: `/v3/projects/75f38533dcd549aa8226b539e83715b4/users/7bf23c2b720b44b5954b80941443a49c/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.375140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73d088-4c3e-4af5-9cf6-2b017082f8f7 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:54.394883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f051a7e-36a9-4e93-8344-720c53bc62f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f38533dcd549aa8226b539e83715b4, user_id=7bf23c2b720b44b5954b80941443a49c, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:54.398042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f051a7e-36a9-4e93-8344-720c53bc62f5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:54.522943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1029/2028] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:54 2022] PUT /identity/v3/projects/75f38533dcd549aa8226b539e83715b4/users/7bf23c2b720b44b5954b80941443a49c/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:54.524398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3a8e9-d988-4ae9-a71d-f5fd2acdf9f1 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.524843 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3a8e9-d988-4ae9-a71d-f5fd2acdf9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.525225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3a8e9-d988-4ae9-a71d-f5fd2acdf9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.533297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1000/2029] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:54 2022] POST /identity/v3/projects => generated 353 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:54.534940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743c19a-7655-47a7-a4a3-1515610bbe7a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.535068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743c19a-7655-47a7-a4a3-1515610bbe7a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.535382 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743c19a-7655-47a7-a4a3-1515610bbe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.551429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7fd3a8e9-d988-4ae9-a71d-f5fd2acdf9f1 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:54.552768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7fd3a8e9-d988-4ae9-a71d-f5fd2acdf9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:54.571961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3743c19a-7655-47a7-a4a3-1515610bbe7a None None] MFA Rules not processed for user `7bf23c2b720b44b5954b80941443a49c`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:54.573615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3743c19a-7655-47a7-a4a3-1515610bbe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:54.621995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1030/2030] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:54.621995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3a8e9-d988-4ae9-a71d-f5fd2acdf9f1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:54.665147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1001/2031] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:54.666163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743c19a-7655-47a7-a4a3-1515610bbe7a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:54.683262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743c19a-7655-47a7-a4a3-1515610bbe7a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:54.683262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cfad4-378b-4b65-95af-e0869719ffed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:54.684219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cfad4-378b-4b65-95af-e0869719ffed admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.684219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cfad4-378b-4b65-95af-e0869719ffed admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.684219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cfad4-378b-4b65-95af-e0869719ffed admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.685326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2cfad4-378b-4b65-95af-e0869719ffed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:54.686757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2cfad4-378b-4b65-95af-e0869719ffed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:54.688913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2cfad4-378b-4b65-95af-e0869719ffed admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:54.696496 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3a8e9-d988-4ae9-a71d-f5fd2acdf9f1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:54.700001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d9096-fb96-4515-bba4-06561aa20a41 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e51e92dde3994b019f485da92f595788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de36d10b6d445dba3259fda83ecbe68', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:54.700763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d9096-fb96-4515-bba4-06561aa20a41 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.701055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d9096-fb96-4515-bba4-06561aa20a41 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.701332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d9096-fb96-4515-bba4-06561aa20a41 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.705449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1002/2032] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:54.705615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d9096-fb96-4515-bba4-06561aa20a41 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:54.705615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d9096-fb96-4515-bba4-06561aa20a41 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:54.705780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cfad4-378b-4b65-95af-e0869719ffed admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:54.705856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d9096-fb96-4515-bba4-06561aa20a41 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:54.705927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1a2cfad4-378b-4b65-95af-e0869719ffed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:54.721595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1031/2033] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:54.722516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d9096-fb96-4515-bba4-06561aa20a41 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:54.733689 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d9096-fb96-4515-bba4-06561aa20a41 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:54.743636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33dd9b-a526-4d31-b44e-5b82fe0c636c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:54.744487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33dd9b-a526-4d31-b44e-5b82fe0c636c admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.744886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33dd9b-a526-4d31-b44e-5b82fe0c636c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.745233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33dd9b-a526-4d31-b44e-5b82fe0c636c admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.746275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33dd9b-a526-4d31-b44e-5b82fe0c636c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:54.748390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33dd9b-a526-4d31-b44e-5b82fe0c636c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:54.832381 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cfad4-378b-4b65-95af-e0869719ffed admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:54.836353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c336a01-e319-4cda-b31c-cd16661ad45c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a58f9101f2d4785b96f0cc55be4330b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:54.836941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c336a01-e319-4cda-b31c-cd16661ad45c service glance] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.842129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c336a01-e319-4cda-b31c-cd16661ad45c service glance] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.842129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c336a01-e319-4cda-b31c-cd16661ad45c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.846616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c336a01-e319-4cda-b31c-cd16661ad45c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:54.848026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c336a01-e319-4cda-b31c-cd16661ad45c service glance] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:54.856934 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1003/2034] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:54.857857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c336a01-e319-4cda-b31c-cd16661ad45c service glance] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:54.869427 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c336a01-e319-4cda-b31c-cd16661ad45c service glance] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:54.874125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e51e92dde3994b019f485da92f595788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de36d10b6d445dba3259fda83ecbe68', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:54.879027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.879027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.879027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] PATH_INFO: `/v3/projects/2af47aaa20e34ad9b797cee743b9e87c` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.879027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2af47aaa20e34ad9b797cee743b9e87c)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:54.879372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:54.912202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1032/2035] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:54 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:54.912202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33dd9b-a526-4d31-b44e-5b82fe0c636c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:54.925072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ab33dd9b-a526-4d31-b44e-5b82fe0c636c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:54.977788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33dd9b-a526-4d31-b44e-5b82fe0c636c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:54.982128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4317a-4d42-4c88-84f2-c6fd7fcb328e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:54.982738 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4317a-4d42-4c88-84f2-c6fd7fcb328e service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:54.983022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4317a-4d42-4c88-84f2-c6fd7fcb328e service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:54.983296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4317a-4d42-4c88-84f2-c6fd7fcb328e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:54.986409 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bcf4317a-4d42-4c88-84f2-c6fd7fcb328e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:55.041854 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af47aaa20e34ad9b797cee743b9e87c', 'request_id': 'req-b8000b3f-54a7-429e-8c74-ec24e771c652'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:55.050016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af47aaa20e34ad9b797cee743b9e87c', 'request_id': 'req-b8000b3f-54a7-429e-8c74-ec24e771c652'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:55.060945 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af47aaa20e34ad9b797cee743b9e87c', 'request_id': 'req-b8000b3f-54a7-429e-8c74-ec24e771c652'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:55.062152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4317a-4d42-4c88-84f2-c6fd7fcb328e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:55.064335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4317a-4d42-4c88-84f2-c6fd7fcb328e service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:55.075251 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1033/2036] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:55.076440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4317a-4d42-4c88-84f2-c6fd7fcb328e service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:55.150333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4317a-4d42-4c88-84f2-c6fd7fcb328e service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:55.211711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] The token cache is being invalidate because project 2af47aaa20e34ad9b797cee743b9e87c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:55.212033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:55.212724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:55.214561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1004/2037] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:54 2022] DELETE /identity/v3/projects/2af47aaa20e34ad9b797cee743b9e87c => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:55.215792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b950e7-6694-491c-84a9-81051e3f2ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:55.216749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b950e7-6694-491c-84a9-81051e3f2ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:55.217177 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b950e7-6694-491c-84a9-81051e3f2ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:55.217515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b950e7-6694-491c-84a9-81051e3f2ac8 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:55.218222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b950e7-6694-491c-84a9-81051e3f2ac8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:55.220629 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b950e7-6694-491c-84a9-81051e3f2ac8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:55.323609 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1034/2038] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:55 2022] POST /identity/v3/users => generated 287 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:55.325237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:55.326963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:55.349181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8000b3f-54a7-429e-8c74-ec24e771c652 tempest-VolumeQuotaClassesTest-1719499983 tempest-VolumeQuotaClassesTest-1719499983-project-admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:55.364470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ec04a-70a3-4e59-9d6e-82ddb8d22d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:55.365110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ec04a-70a3-4e59-9d6e-82ddb8d22d21 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:55.365395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ec04a-70a3-4e59-9d6e-82ddb8d22d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:55.365700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ec04a-70a3-4e59-9d6e-82ddb8d22d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:55.366370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ec04a-70a3-4e59-9d6e-82ddb8d22d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:55.368174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ec04a-70a3-4e59-9d6e-82ddb8d22d21 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:55.384362 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1005/2039] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:55.386286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b950e7-6694-491c-84a9-81051e3f2ac8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:55.408618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b950e7-6694-491c-84a9-81051e3f2ac8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:55.423960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981876db-4d05-467d-8d98-7dc3484be040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:55.425358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981876db-4d05-467d-8d98-7dc3484be040 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:55.425776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981876db-4d05-467d-8d98-7dc3484be040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:55.426699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981876db-4d05-467d-8d98-7dc3484be040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:55.428002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981876db-4d05-467d-8d98-7dc3484be040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:55.430194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981876db-4d05-467d-8d98-7dc3484be040 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:55.442425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1035/2040] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:55.446726 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ec04a-70a3-4e59-9d6e-82ddb8d22d21 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:55.458675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ec04a-70a3-4e59-9d6e-82ddb8d22d21 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:55.463832 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7aa503-c3ec-4530-be2f-48603489dfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:55.464500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7aa503-c3ec-4530-be2f-48603489dfe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:55.465128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7aa503-c3ec-4530-be2f-48603489dfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:55.465128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7aa503-c3ec-4530-be2f-48603489dfe7 admin admin] PATH_INFO: `/v3/projects/9a0c205d9f1544c8bd1d9d346d5c50af/users/3bae41894e5f499fa5b18c24389a06b3/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:55.482358 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7aa503-c3ec-4530-be2f-48603489dfe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0c205d9f1544c8bd1d9d346d5c50af, user_id=3bae41894e5f499fa5b18c24389a06b3, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:55.485808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7aa503-c3ec-4530-be2f-48603489dfe7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:55.583284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1006/2041] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:55 2022] PUT /identity/v3/projects/9a0c205d9f1544c8bd1d9d346d5c50af/users/3bae41894e5f499fa5b18c24389a06b3/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:55.593623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d546d5-a287-45cb-9c60-d9d4239d16dc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:55.593912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d546d5-a287-45cb-9c60-d9d4239d16dc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:55.594203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d546d5-a287-45cb-9c60-d9d4239d16dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:55.629778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e8d546d5-a287-45cb-9c60-d9d4239d16dc None None] MFA Rules not processed for user `3bae41894e5f499fa5b18c24389a06b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:55.631865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e8d546d5-a287-45cb-9c60-d9d4239d16dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:55.645411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:55.707959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1036/2042] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:55.948201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7aa503-c3ec-4530-be2f-48603489dfe7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:55.953869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bd7aa503-c3ec-4530-be2f-48603489dfe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:56.057351 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:56.074645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7aa503-c3ec-4530-be2f-48603489dfe7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:56.079016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db0922-fe9b-489e-b2d8-4c4d89aa0fa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:56.079892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db0922-fe9b-489e-b2d8-4c4d89aa0fa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:56.080179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db0922-fe9b-489e-b2d8-4c4d89aa0fa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:56.080449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db0922-fe9b-489e-b2d8-4c4d89aa0fa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:56.089727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db0922-fe9b-489e-b2d8-4c4d89aa0fa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:56.091279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db0922-fe9b-489e-b2d8-4c4d89aa0fa6 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:56.100668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1007/2043] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:56.128265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d546d5-a287-45cb-9c60-d9d4239d16dc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:56.199245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d546d5-a287-45cb-9c60-d9d4239d16dc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:56.204015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53115f2-5394-4a2f-8f62-f68a5c5ec82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:56.204649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53115f2-5394-4a2f-8f62-f68a5c5ec82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:56.204933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53115f2-5394-4a2f-8f62-f68a5c5ec82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:56.205203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53115f2-5394-4a2f-8f62-f68a5c5ec82f admin admin] PATH_INFO: `/v3/users/3bae41894e5f499fa5b18c24389a06b3` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:56.206370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53115f2-5394-4a2f-8f62-f68a5c5ec82f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bae41894e5f499fa5b18c24389a06b3)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:56.208310 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53115f2-5394-4a2f-8f62-f68a5c5ec82f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:56.286585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db0922-fe9b-489e-b2d8-4c4d89aa0fa6 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:56.288844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-58db0922-fe9b-489e-b2d8-4c4d89aa0fa6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:56.308191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db0922-fe9b-489e-b2d8-4c4d89aa0fa6 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:56.312929 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8812f-83e8-4b76-8f49-3fd2db51a8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:56.314249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8812f-83e8-4b76-8f49-3fd2db51a8f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:56.314249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8812f-83e8-4b76-8f49-3fd2db51a8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:56.314249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8812f-83e8-4b76-8f49-3fd2db51a8f6 admin admin] PATH_INFO: `/v3/users/56a8c5b85a4845008533fff3a8ab5675` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:56.315197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8812f-83e8-4b76-8f49-3fd2db51a8f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a8c5b85a4845008533fff3a8ab5675)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:56.317242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8812f-83e8-4b76-8f49-3fd2db51a8f6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:56.374854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d53115f2-5394-4a2f-8f62-f68a5c5ec82f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bae41894e5f499fa5b18c24389a06b3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:56.445411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d53115f2-5394-4a2f-8f62-f68a5c5ec82f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bae41894e5f499fa5b18c24389a06b3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:56.446133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d53115f2-5394-4a2f-8f62-f68a5c5ec82f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bae41894e5f499fa5b18c24389a06b3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:56.457187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d53115f2-5394-4a2f-8f62-f68a5c5ec82f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bae41894e5f499fa5b18c24389a06b3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:56.469815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d53115f2-5394-4a2f-8f62-f68a5c5ec82f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bae41894e5f499fa5b18c24389a06b3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:56.477045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1037/2044] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:56 2022] DELETE /identity/v3/users/3bae41894e5f499fa5b18c24389a06b3 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:56.483109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d503ff-1529-416c-b5bc-06cd993e50c4 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:56.483109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d503ff-1529-416c-b5bc-06cd993e50c4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:56.483109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d503ff-1529-416c-b5bc-06cd993e50c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:56.509953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a2d503ff-1529-416c-b5bc-06cd993e50c4 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:56.511782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a2d503ff-1529-416c-b5bc-06cd993e50c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:56.558994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f6b8812f-83e8-4b76-8f49-3fd2db51a8f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a8c5b85a4845008533fff3a8ab5675'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:56.613452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1038/2045] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:56.627315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d503ff-1529-416c-b5bc-06cd993e50c4 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:56.627513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f6b8812f-83e8-4b76-8f49-3fd2db51a8f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a8c5b85a4845008533fff3a8ab5675'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:56.628329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f6b8812f-83e8-4b76-8f49-3fd2db51a8f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a8c5b85a4845008533fff3a8ab5675'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:56.629771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a2d503ff-1529-416c-b5bc-06cd993e50c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:56.640347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f6b8812f-83e8-4b76-8f49-3fd2db51a8f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a8c5b85a4845008533fff3a8ab5675'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:56.659728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f6b8812f-83e8-4b76-8f49-3fd2db51a8f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a8c5b85a4845008533fff3a8ab5675'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:56.678891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1008/2046] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:56 2022] DELETE /identity/v3/users/56a8c5b85a4845008533fff3a8ab5675 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:56.685973 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9618b8f-ad59-4fc5-b509-7002f5788d45 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:56.686340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9618b8f-ad59-4fc5-b509-7002f5788d45 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:56.686626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9618b8f-ad59-4fc5-b509-7002f5788d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:56.708547 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e9618b8f-ad59-4fc5-b509-7002f5788d45 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:56.710083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e9618b8f-ad59-4fc5-b509-7002f5788d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:56.711717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d503ff-1529-416c-b5bc-06cd993e50c4 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:56.789852 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1726bd-b2e9-4681-a0fe-b793cbba4908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:56.790501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1726bd-b2e9-4681-a0fe-b793cbba4908 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:56.790955 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1726bd-b2e9-4681-a0fe-b793cbba4908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:56.791227 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1726bd-b2e9-4681-a0fe-b793cbba4908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:56.793056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cc1726bd-b2e9-4681-a0fe-b793cbba4908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:56.795846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1009/2047] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:56.805908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9618b8f-ad59-4fc5-b509-7002f5788d45 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:56.806503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1726bd-b2e9-4681-a0fe-b793cbba4908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:56.807288 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e9618b8f-ad59-4fc5-b509-7002f5788d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:56.808075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1726bd-b2e9-4681-a0fe-b793cbba4908 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:56.819517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1039/2048] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:56.819735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9618b8f-ad59-4fc5-b509-7002f5788d45 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:56.823528 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123ef90-3d37-4ce9-bdcd-e87b342a0852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:56.831438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123ef90-3d37-4ce9-bdcd-e87b342a0852 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:56.831438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123ef90-3d37-4ce9-bdcd-e87b342a0852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:56.831438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123ef90-3d37-4ce9-bdcd-e87b342a0852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:56.835081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123ef90-3d37-4ce9-bdcd-e87b342a0852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:56.836519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123ef90-3d37-4ce9-bdcd-e87b342a0852 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:56.847289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1010/2049] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:56.867705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:57.503574 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1726bd-b2e9-4681-a0fe-b793cbba4908 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:57.505206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cc1726bd-b2e9-4681-a0fe-b793cbba4908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:57.519003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1726bd-b2e9-4681-a0fe-b793cbba4908 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:57.523696 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a7c5b-85f6-4d9d-985c-9183d9054c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:57.524334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a7c5b-85f6-4d9d-985c-9183d9054c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:57.524607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a7c5b-85f6-4d9d-985c-9183d9054c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:57.524872 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a7c5b-85f6-4d9d-985c-9183d9054c78 admin admin] PATH_INFO: `/v3/projects/9a0c205d9f1544c8bd1d9d346d5c50af` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:57.526851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a7c5b-85f6-4d9d-985c-9183d9054c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a0c205d9f1544c8bd1d9d346d5c50af)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:57.529870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a7c5b-85f6-4d9d-985c-9183d9054c78 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:57.544946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123ef90-3d37-4ce9-bdcd-e87b342a0852 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:57.556597 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7123ef90-3d37-4ce9-bdcd-e87b342a0852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:57.562715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123ef90-3d37-4ce9-bdcd-e87b342a0852 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:57.568403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:57.569048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:57.569330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:57.569648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] PATH_INFO: `/v3/projects/3358298e9a2c426d86d0b22068b83146` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:57.570807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] RBAC: Authorizing `identity:delete_project(project_id=3358298e9a2c426d86d0b22068b83146)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:57.572813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:57.731733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:57.742301 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d77a7c5b-85f6-4d9d-985c-9183d9054c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a0c205d9f1544c8bd1d9d346d5c50af', 'request_id': 'req-d77a7c5b-85f6-4d9d-985c-9183d9054c78'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:57.752687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d77a7c5b-85f6-4d9d-985c-9183d9054c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a0c205d9f1544c8bd1d9d346d5c50af', 'request_id': 'req-d77a7c5b-85f6-4d9d-985c-9183d9054c78'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:57.761674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d77a7c5b-85f6-4d9d-985c-9183d9054c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a0c205d9f1544c8bd1d9d346d5c50af', 'request_id': 'req-d77a7c5b-85f6-4d9d-985c-9183d9054c78'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:57.774118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3358298e9a2c426d86d0b22068b83146', 'request_id': 'req-77d3f079-1447-406c-babe-7210c59dafae'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:57.786547 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3358298e9a2c426d86d0b22068b83146', 'request_id': 'req-77d3f079-1447-406c-babe-7210c59dafae'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:57.798834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3358298e9a2c426d86d0b22068b83146', 'request_id': 'req-77d3f079-1447-406c-babe-7210c59dafae'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:57.839025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d77a7c5b-85f6-4d9d-985c-9183d9054c78 admin admin] The token cache is being invalidate because project 9a0c205d9f1544c8bd1d9d346d5c50af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:57.839332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d77a7c5b-85f6-4d9d-985c-9183d9054c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:57.840148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d77a7c5b-85f6-4d9d-985c-9183d9054c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:57.842602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1040/2050] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:57 2022] DELETE /identity/v3/projects/9a0c205d9f1544c8bd1d9d346d5c50af => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:57.846925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829c4fc-59eb-45c5-8dc9-ec0a50a4b2d7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:57.847209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829c4fc-59eb-45c5-8dc9-ec0a50a4b2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:57.847489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829c4fc-59eb-45c5-8dc9-ec0a50a4b2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:57.873881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2829c4fc-59eb-45c5-8dc9-ec0a50a4b2d7 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:57.875630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2829c4fc-59eb-45c5-8dc9-ec0a50a4b2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:57.913214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] The token cache is being invalidate because project 3358298e9a2c426d86d0b22068b83146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:57.913625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:57.914634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:57.916997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1011/2051] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:57 2022] DELETE /identity/v3/projects/3358298e9a2c426d86d0b22068b83146 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:57.922476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:57.924256 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:57.960230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1041/2052] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:57.969900 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09e9e0-178b-4688-8345-da55c7268b53 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:57.970278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09e9e0-178b-4688-8345-da55c7268b53 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:57.970671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09e9e0-178b-4688-8345-da55c7268b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:57.994343 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4d09e9e0-178b-4688-8345-da55c7268b53 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:57.995577 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4d09e9e0-178b-4688-8345-da55c7268b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:58.002665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3f079-1447-406c-babe-7210c59dafae admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:58.005057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:58.005719 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1042/2053] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:58.006179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:58.006571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:58.007040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] PATH_INFO: `/v3/users/e51e92dde3994b019f485da92f595788` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:58.009202 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e51e92dde3994b019f485da92f595788)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:58.010273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09e9e0-178b-4688-8345-da55c7268b53 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:58.013051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:58.020418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09e9e0-178b-4688-8345-da55c7268b53 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:58.029735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c794bf0-b5be-47e5-bdeb-3e39bcb038ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:58.037011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c794bf0-b5be-47e5-bdeb-3e39bcb038ec admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:58.037011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c794bf0-b5be-47e5-bdeb-3e39bcb038ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:58.037011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c794bf0-b5be-47e5-bdeb-3e39bcb038ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:58.037011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c794bf0-b5be-47e5-bdeb-3e39bcb038ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:58.037011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c794bf0-b5be-47e5-bdeb-3e39bcb038ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:58.037011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c794bf0-b5be-47e5-bdeb-3e39bcb038ec admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:58.051430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1043/2054] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:49:58.052517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c794bf0-b5be-47e5-bdeb-3e39bcb038ec admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:58.065893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:58.066650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c794bf0-b5be-47e5-bdeb-3e39bcb038ec admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:58.071857 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d06c77-64a7-4c3d-8b72-8ff2c41e4439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:58.072502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06c77-64a7-4c3d-8b72-8ff2c41e4439 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:58.072781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06c77-64a7-4c3d-8b72-8ff2c41e4439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:58.073052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06c77-64a7-4c3d-8b72-8ff2c41e4439 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:58.073710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d06c77-64a7-4c3d-8b72-8ff2c41e4439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:58.075636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d06c77-64a7-4c3d-8b72-8ff2c41e4439 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:58.183799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1044/2055] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:58 2022] POST /identity/v3/projects => generated 359 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:58.190888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d06c77-64a7-4c3d-8b72-8ff2c41e4439 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:58.258860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d06c77-64a7-4c3d-8b72-8ff2c41e4439 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:58.273608 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa993e1c-fde0-4f7e-899e-89276cc0f878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:58.273965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa993e1c-fde0-4f7e-899e-89276cc0f878 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:58.274317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa993e1c-fde0-4f7e-899e-89276cc0f878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:58.275042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa993e1c-fde0-4f7e-899e-89276cc0f878 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:58.276235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa993e1c-fde0-4f7e-899e-89276cc0f878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:58.279620 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa993e1c-fde0-4f7e-899e-89276cc0f878 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:58.285153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e51e92dde3994b019f485da92f595788'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:58.381944 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e51e92dde3994b019f485da92f595788'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:58.382531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e51e92dde3994b019f485da92f595788'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:58.387973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1045/2056] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:58 2022] POST /identity/v3/users => generated 280 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:58.390362 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e51e92dde3994b019f485da92f595788'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:58.393009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa993e1c-fde0-4f7e-899e-89276cc0f878 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:58.395405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-aa993e1c-fde0-4f7e-899e-89276cc0f878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:58.404605 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e51e92dde3994b019f485da92f595788'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:58.412317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1012/2057] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:57 2022] DELETE /identity/v3/users/e51e92dde3994b019f485da92f595788 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:58.413980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa993e1c-fde0-4f7e-899e-89276cc0f878 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:58.492245 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25df215-77c6-4983-b02c-d9e5cc56cf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:58.493319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25df215-77c6-4983-b02c-d9e5cc56cf37 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:58.493796 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25df215-77c6-4983-b02c-d9e5cc56cf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:58.494170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25df215-77c6-4983-b02c-d9e5cc56cf37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:58.495001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25df215-77c6-4983-b02c-d9e5cc56cf37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:58.497924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25df215-77c6-4983-b02c-d9e5cc56cf37 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:58.513764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1046/2058] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:49:58.521933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:58.538684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef8c9d-aeca-41a6-ade5-f1f510523e5a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:58.544114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d340146-ded0-49a6-ade1-760fd7a7ce7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:58.544831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d340146-ded0-49a6-ade1-760fd7a7ce7e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:58.545186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d340146-ded0-49a6-ade1-760fd7a7ce7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:58.545853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d340146-ded0-49a6-ade1-760fd7a7ce7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:58.546187 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d340146-ded0-49a6-ade1-760fd7a7ce7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:58.547894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d340146-ded0-49a6-ade1-760fd7a7ce7e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:58.558894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1013/2059] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:49:58.566064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25df215-77c6-4983-b02c-d9e5cc56cf37 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:58.576754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25df215-77c6-4983-b02c-d9e5cc56cf37 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:58.582094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d86a8-3ece-44de-8a72-476b2111622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:58.582746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d86a8-3ece-44de-8a72-476b2111622f admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:58.583027 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d86a8-3ece-44de-8a72-476b2111622f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:58.583300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d86a8-3ece-44de-8a72-476b2111622f admin admin] PATH_INFO: `/v3/projects/c614fc1df2694c95928caa45cd72e6ae/users/a5d4c5009c3944d5aaa901f9aae9735d/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:58.599795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d86a8-3ece-44de-8a72-476b2111622f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c614fc1df2694c95928caa45cd72e6ae, user_id=a5d4c5009c3944d5aaa901f9aae9735d, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:58.608294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d86a8-3ece-44de-8a72-476b2111622f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:58.720436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1047/2060] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:49:58 2022] PUT /identity/v3/projects/c614fc1df2694c95928caa45cd72e6ae/users/a5d4c5009c3944d5aaa901f9aae9735d/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:58.743795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdf3b7-cbed-470a-81f4-9d23afe3588f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:58.743795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdf3b7-cbed-470a-81f4-9d23afe3588f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:58.743795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdf3b7-cbed-470a-81f4-9d23afe3588f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:58.776482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ffbdf3b7-cbed-470a-81f4-9d23afe3588f None None] MFA Rules not processed for user `a5d4c5009c3944d5aaa901f9aae9735d`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:58.778307 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ffbdf3b7-cbed-470a-81f4-9d23afe3588f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:58.866631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1014/2061] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:49:58.911101 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d86a8-3ece-44de-8a72-476b2111622f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:58.913449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-943d86a8-3ece-44de-8a72-476b2111622f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:58.995966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d86a8-3ece-44de-8a72-476b2111622f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:58.999756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c916d6d-7b60-4b93-896c-a3e95d006190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:59.000961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c916d6d-7b60-4b93-896c-a3e95d006190 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:59.001324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c916d6d-7b60-4b93-896c-a3e95d006190 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:59.001674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c916d6d-7b60-4b93-896c-a3e95d006190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:59.012186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c916d6d-7b60-4b93-896c-a3e95d006190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:59.013809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c916d6d-7b60-4b93-896c-a3e95d006190 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:59.031758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1048/2062] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:49:59.046113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdf3b7-cbed-470a-81f4-9d23afe3588f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:59.123878 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdf3b7-cbed-470a-81f4-9d23afe3588f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:59.129446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c8659-aff2-4611-ba1d-dc59921973e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:59.130214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c8659-aff2-4611-ba1d-dc59921973e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:59.130491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c8659-aff2-4611-ba1d-dc59921973e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:59.130769 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c8659-aff2-4611-ba1d-dc59921973e1 admin admin] PATH_INFO: `/v3/users/a5d4c5009c3944d5aaa901f9aae9735d` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:59.131910 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c8659-aff2-4611-ba1d-dc59921973e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d4c5009c3944d5aaa901f9aae9735d)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:59.134063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c8659-aff2-4611-ba1d-dc59921973e1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:59.173608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:59.242677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c916d6d-7b60-4b93-896c-a3e95d006190 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:59.245113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6c916d6d-7b60-4b93-896c-a3e95d006190 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:59.267048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c916d6d-7b60-4b93-896c-a3e95d006190 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:59.283047 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a7dd6-5758-4c83-bf43-c373097b8be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:59.284163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a7dd6-5758-4c83-bf43-c373097b8be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:59.284163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a7dd6-5758-4c83-bf43-c373097b8be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:59.284417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a7dd6-5758-4c83-bf43-c373097b8be2 admin admin] PATH_INFO: `/v3/projects/7de36d10b6d445dba3259fda83ecbe68` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:59.289517 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a7dd6-5758-4c83-bf43-c373097b8be2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7de36d10b6d445dba3259fda83ecbe68)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:59.293653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a7dd6-5758-4c83-bf43-c373097b8be2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:59.331418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6f7c8659-aff2-4611-ba1d-dc59921973e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d4c5009c3944d5aaa901f9aae9735d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.382352 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6f7c8659-aff2-4611-ba1d-dc59921973e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d4c5009c3944d5aaa901f9aae9735d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.383269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6f7c8659-aff2-4611-ba1d-dc59921973e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d4c5009c3944d5aaa901f9aae9735d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.393885 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6f7c8659-aff2-4611-ba1d-dc59921973e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d4c5009c3944d5aaa901f9aae9735d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.405983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e5a7dd6-5758-4c83-bf43-c373097b8be2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7de36d10b6d445dba3259fda83ecbe68', 'request_id': 'req-8e5a7dd6-5758-4c83-bf43-c373097b8be2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.409614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6f7c8659-aff2-4611-ba1d-dc59921973e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d4c5009c3944d5aaa901f9aae9735d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.410094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:49:59.413939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e5a7dd6-5758-4c83-bf43-c373097b8be2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7de36d10b6d445dba3259fda83ecbe68', 'request_id': 'req-8e5a7dd6-5758-4c83-bf43-c373097b8be2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.417553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1015/2063] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:59 2022] DELETE /identity/v3/users/a5d4c5009c3944d5aaa901f9aae9735d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:49:59.421453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85300f13-5396-48ad-9e9a-a32ca9626b5c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:59.424097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85300f13-5396-48ad-9e9a-a32ca9626b5c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:59.424392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85300f13-5396-48ad-9e9a-a32ca9626b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:59.425408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e5a7dd6-5758-4c83-bf43-c373097b8be2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7de36d10b6d445dba3259fda83ecbe68', 'request_id': 'req-8e5a7dd6-5758-4c83-bf43-c373097b8be2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.459877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-85300f13-5396-48ad-9e9a-a32ca9626b5c None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:59.461134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-85300f13-5396-48ad-9e9a-a32ca9626b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:59.513184 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e5a7dd6-5758-4c83-bf43-c373097b8be2 admin admin] The token cache is being invalidate because project 7de36d10b6d445dba3259fda83ecbe68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:49:59.513398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e5a7dd6-5758-4c83-bf43-c373097b8be2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.514118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e5a7dd6-5758-4c83-bf43-c373097b8be2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.515688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1049/2064] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:49:59 2022] DELETE /identity/v3/projects/7de36d10b6d445dba3259fda83ecbe68 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:49:59.520657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6487c-cdca-4bee-ba81-2210f4f27b39 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:59.520941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6487c-cdca-4bee-ba81-2210f4f27b39 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:59.521221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6487c-cdca-4bee-ba81-2210f4f27b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:59.523819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1016/2065] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:59.533803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85300f13-5396-48ad-9e9a-a32ca9626b5c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:59.535342 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-85300f13-5396-48ad-9e9a-a32ca9626b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:59.541226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-40f6487c-cdca-4bee-ba81-2210f4f27b39 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:59.547444 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-40f6487c-cdca-4bee-ba81-2210f4f27b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:59.555685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1050/2066] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:59.564165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197c4f3-b6fd-4305-8206-c66a67ab7dc3 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:59.564495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197c4f3-b6fd-4305-8206-c66a67ab7dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:59.564766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197c4f3-b6fd-4305-8206-c66a67ab7dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:59.597759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0197c4f3-b6fd-4305-8206-c66a67ab7dc3 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:49:59.599053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0197c4f3-b6fd-4305-8206-c66a67ab7dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:59.606334 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85300f13-5396-48ad-9e9a-a32ca9626b5c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:59.626492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0802341-68fb-4292-b53e-2b0300fefd1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:59.628410 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0802341-68fb-4292-b53e-2b0300fefd1c service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:59.629031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0802341-68fb-4292-b53e-2b0300fefd1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:59.629659 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0802341-68fb-4292-b53e-2b0300fefd1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:59.641422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1051/2067] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:49:59.643751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0802341-68fb-4292-b53e-2b0300fefd1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:59.645731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0802341-68fb-4292-b53e-2b0300fefd1c service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:59.647427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197c4f3-b6fd-4305-8206-c66a67ab7dc3 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:59.660004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1017/2068] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:49:59.661414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0802341-68fb-4292-b53e-2b0300fefd1c service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:59.663884 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197c4f3-b6fd-4305-8206-c66a67ab7dc3 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:59.664385 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b0802341-68fb-4292-b53e-2b0300fefd1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:49:59.669099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6520a18-8a33-4c41-910e-4a7a30d6ff9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:59.669767 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6520a18-8a33-4c41-910e-4a7a30d6ff9d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:59.670049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6520a18-8a33-4c41-910e-4a7a30d6ff9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:59.670319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6520a18-8a33-4c41-910e-4a7a30d6ff9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:59.677402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6520a18-8a33-4c41-910e-4a7a30d6ff9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:49:59.677402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6520a18-8a33-4c41-910e-4a7a30d6ff9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:59.677402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6520a18-8a33-4c41-910e-4a7a30d6ff9d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:59.681132 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0802341-68fb-4292-b53e-2b0300fefd1c service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:59.686585 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1052/2069] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:49:59.690673 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ac49a-8c34-43c9-8160-31718473cc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:59.692800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6520a18-8a33-4c41-910e-4a7a30d6ff9d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:59.694035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ac49a-8c34-43c9-8160-31718473cc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:59.694318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ac49a-8c34-43c9-8160-31718473cc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:59.694594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ac49a-8c34-43c9-8160-31718473cc50 admin admin] PATH_INFO: `/v3/users/2ed45d354d5a4b7ca8ea6839307e44a7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:59.695865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ac49a-8c34-43c9-8160-31718473cc50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed45d354d5a4b7ca8ea6839307e44a7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:59.697853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ac49a-8c34-43c9-8160-31718473cc50 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:59.709901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6520a18-8a33-4c41-910e-4a7a30d6ff9d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:59.716627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758968fb-7e67-4522-820f-8aed7f53b126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:59.717722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758968fb-7e67-4522-820f-8aed7f53b126 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:59.718119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758968fb-7e67-4522-820f-8aed7f53b126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:59.719183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758968fb-7e67-4522-820f-8aed7f53b126 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:59.720114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758968fb-7e67-4522-820f-8aed7f53b126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:59.722697 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758968fb-7e67-4522-820f-8aed7f53b126 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:59.823755 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1053/2070] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:49:59 2022] POST /identity/v3/projects => generated 399 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:49:59.827841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758968fb-7e67-4522-820f-8aed7f53b126 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:49:59.887935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1c2ac49a-8c34-43c9-8160-31718473cc50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed45d354d5a4b7ca8ea6839307e44a7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.908302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758968fb-7e67-4522-820f-8aed7f53b126 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:49:59.913216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a2438-aa35-46b6-8ff0-73cddf678113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:49:59.913858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a2438-aa35-46b6-8ff0-73cddf678113 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:49:59.914170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a2438-aa35-46b6-8ff0-73cddf678113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:49:59.914475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a2438-aa35-46b6-8ff0-73cddf678113 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:49:59.915102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a2438-aa35-46b6-8ff0-73cddf678113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:49:59.917271 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a2438-aa35-46b6-8ff0-73cddf678113 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:49:59.982081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1c2ac49a-8c34-43c9-8160-31718473cc50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed45d354d5a4b7ca8ea6839307e44a7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.982702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1c2ac49a-8c34-43c9-8160-31718473cc50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed45d354d5a4b7ca8ea6839307e44a7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:49:59.990505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1c2ac49a-8c34-43c9-8160-31718473cc50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed45d354d5a4b7ca8ea6839307e44a7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:00.004248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1c2ac49a-8c34-43c9-8160-31718473cc50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed45d354d5a4b7ca8ea6839307e44a7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:00.010617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1018/2071] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:49:59 2022] DELETE /identity/v3/users/2ed45d354d5a4b7ca8ea6839307e44a7 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:00.015104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaab1e-05dc-4cfd-8906-495e85ecefaf None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:00.015509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaab1e-05dc-4cfd-8906-495e85ecefaf None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:00.016350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaab1e-05dc-4cfd-8906-495e85ecefaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:00.044634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ecbaab1e-05dc-4cfd-8906-495e85ecefaf None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:00.045969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ecbaab1e-05dc-4cfd-8906-495e85ecefaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:00.064754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1054/2072] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:49:59 2022] POST /identity/v3/users => generated 300 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:50:00.068704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a2438-aa35-46b6-8ff0-73cddf678113 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:00.075595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-105a2438-aa35-46b6-8ff0-73cddf678113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:00.099553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a2438-aa35-46b6-8ff0-73cddf678113 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:00.146196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1019/2073] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:50:00.157636 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaab1e-05dc-4cfd-8906-495e85ecefaf None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:00.159207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ecbaab1e-05dc-4cfd-8906-495e85ecefaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:00.171404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47b418-5921-4606-a21b-57282b61137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:00.172821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47b418-5921-4606-a21b-57282b61137c admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:00.175121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47b418-5921-4606-a21b-57282b61137c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:00.175121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47b418-5921-4606-a21b-57282b61137c admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:00.176178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47b418-5921-4606-a21b-57282b61137c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:00.177856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47b418-5921-4606-a21b-57282b61137c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:00.192539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1055/2074] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:00.194490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47b418-5921-4606-a21b-57282b61137c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:00.205587 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47b418-5921-4606-a21b-57282b61137c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:00.210615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef17696-2521-451e-9c42-4e5f41452be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:00.211294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef17696-2521-451e-9c42-4e5f41452be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:00.211691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef17696-2521-451e-9c42-4e5f41452be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:00.212026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef17696-2521-451e-9c42-4e5f41452be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:00.212805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef17696-2521-451e-9c42-4e5f41452be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:00.214881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef17696-2521-451e-9c42-4e5f41452be5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:00.232913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaab1e-05dc-4cfd-8906-495e85ecefaf None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:00.233086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1056/2075] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:00.233182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ce6aa-5059-4d30-a4c3-f45be78b1161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:00.233182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ce6aa-5059-4d30-a4c3-f45be78b1161 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:00.233182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ce6aa-5059-4d30-a4c3-f45be78b1161 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:00.233182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ce6aa-5059-4d30-a4c3-f45be78b1161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:00.242823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ce6aa-5059-4d30-a4c3-f45be78b1161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:00.244300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ce6aa-5059-4d30-a4c3-f45be78b1161 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:00.244434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef17696-2521-451e-9c42-4e5f41452be5 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:00.254319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef17696-2521-451e-9c42-4e5f41452be5 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:00.254867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1020/2076] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:50:00.259073 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db03d2-6c49-4ea7-acd8-510aef7c9f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:00.259765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db03d2-6c49-4ea7-acd8-510aef7c9f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:00.260016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db03d2-6c49-4ea7-acd8-510aef7c9f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:00.260714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db03d2-6c49-4ea7-acd8-510aef7c9f6c admin admin] PATH_INFO: `/v3/projects/931be0e8324348acaa6bbbf3b7481ff6/users/e695fecba1f84058a9d6057e5a9a9095/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:00.275216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ce6aa-5059-4d30-a4c3-f45be78b1161 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:00.277027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-556ce6aa-5059-4d30-a4c3-f45be78b1161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:00.279988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db03d2-6c49-4ea7-acd8-510aef7c9f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=931be0e8324348acaa6bbbf3b7481ff6, user_id=e695fecba1f84058a9d6057e5a9a9095, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:00.283939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db03d2-6c49-4ea7-acd8-510aef7c9f6c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:00.292638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ce6aa-5059-4d30-a4c3-f45be78b1161 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:00.298123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eac291-9575-42fe-b37d-a257253ebf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:00.303409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eac291-9575-42fe-b37d-a257253ebf79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:00.303409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eac291-9575-42fe-b37d-a257253ebf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:00.303409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eac291-9575-42fe-b37d-a257253ebf79 admin admin] PATH_INFO: `/v3/projects/c614fc1df2694c95928caa45cd72e6ae` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:00.303409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eac291-9575-42fe-b37d-a257253ebf79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c614fc1df2694c95928caa45cd72e6ae)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:00.305794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eac291-9575-42fe-b37d-a257253ebf79 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:00.347060 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:00.411756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1057/2077] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:00 2022] PUT /identity/v3/projects/931be0e8324348acaa6bbbf3b7481ff6/users/e695fecba1f84058a9d6057e5a9a9095/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:50:00.425069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcf202-5671-4400-88ab-0365fe1c8703 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:00.425363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcf202-5671-4400-88ab-0365fe1c8703 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:00.425724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcf202-5671-4400-88ab-0365fe1c8703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:00.461866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-55eac291-9575-42fe-b37d-a257253ebf79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c614fc1df2694c95928caa45cd72e6ae', 'request_id': 'req-55eac291-9575-42fe-b37d-a257253ebf79'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:00.472985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-55eac291-9575-42fe-b37d-a257253ebf79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c614fc1df2694c95928caa45cd72e6ae', 'request_id': 'req-55eac291-9575-42fe-b37d-a257253ebf79'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:00.480737 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-55eac291-9575-42fe-b37d-a257253ebf79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c614fc1df2694c95928caa45cd72e6ae', 'request_id': 'req-55eac291-9575-42fe-b37d-a257253ebf79'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:00.482257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e9bcf202-5671-4400-88ab-0365fe1c8703 None None] MFA Rules not processed for user `e695fecba1f84058a9d6057e5a9a9095`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:00.485181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e9bcf202-5671-4400-88ab-0365fe1c8703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:00.513831 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:00.579138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-55eac291-9575-42fe-b37d-a257253ebf79 admin admin] The token cache is being invalidate because project c614fc1df2694c95928caa45cd72e6ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:50:00.580205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-55eac291-9575-42fe-b37d-a257253ebf79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:00.581495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-55eac291-9575-42fe-b37d-a257253ebf79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:00.588727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1021/2078] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:50:00 2022] DELETE /identity/v3/projects/c614fc1df2694c95928caa45cd72e6ae => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:00.595642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747993f0-2119-46de-b52a-b42ddbc8b4d5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:00.596116 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747993f0-2119-46de-b52a-b42ddbc8b4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:00.596581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747993f0-2119-46de-b52a-b42ddbc8b4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:00.646290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1058/2079] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:00.646290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcf202-5671-4400-88ab-0365fe1c8703 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:00.646290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e9bcf202-5671-4400-88ab-0365fe1c8703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:00.659753 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-747993f0-2119-46de-b52a-b42ddbc8b4d5 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:00.661750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-747993f0-2119-46de-b52a-b42ddbc8b4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:00.755612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcf202-5671-4400-88ab-0365fe1c8703 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:00.762125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c92116-807c-48ba-8112-d2045f7d8fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:00.762318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c92116-807c-48ba-8112-d2045f7d8fcc admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:00.762526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c92116-807c-48ba-8112-d2045f7d8fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:00.762804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c92116-807c-48ba-8112-d2045f7d8fcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:00.763445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c92116-807c-48ba-8112-d2045f7d8fcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:00.765648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c92116-807c-48ba-8112-d2045f7d8fcc admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:00.782539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:00.786639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1022/2080] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:50:00.793455 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966f784-aa6a-4edb-a9f4-5441515d4317 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:00.794045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966f784-aa6a-4edb-a9f4-5441515d4317 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:00.794323 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966f784-aa6a-4edb-a9f4-5441515d4317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:00.817700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4966f784-aa6a-4edb-a9f4-5441515d4317 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:00.818942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4966f784-aa6a-4edb-a9f4-5441515d4317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:00.827053 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1023/2081] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:50:00.831579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966f784-aa6a-4edb-a9f4-5441515d4317 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:00.841244 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966f784-aa6a-4edb-a9f4-5441515d4317 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:00.846555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ddf03-c07a-4950-a23e-9f1c09a734f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:00.847171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ddf03-c07a-4950-a23e-9f1c09a734f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:00.847450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ddf03-c07a-4950-a23e-9f1c09a734f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:00.847722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ddf03-c07a-4950-a23e-9f1c09a734f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:00.848376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ddf03-c07a-4950-a23e-9f1c09a734f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:50:00.848678 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ddf03-c07a-4950-a23e-9f1c09a734f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:00.850300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ddf03-c07a-4950-a23e-9f1c09a734f3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:00.863192 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1024/2082] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:50:00.867693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ddf03-c07a-4950-a23e-9f1c09a734f3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:00.878982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ddf03-c07a-4950-a23e-9f1c09a734f3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:00.883776 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82489ce7-56b6-4cb9-bf48-22751100ee85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:00.884400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82489ce7-56b6-4cb9-bf48-22751100ee85 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:00.884681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82489ce7-56b6-4cb9-bf48-22751100ee85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:00.885054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82489ce7-56b6-4cb9-bf48-22751100ee85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:00.885819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82489ce7-56b6-4cb9-bf48-22751100ee85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:00.887877 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82489ce7-56b6-4cb9-bf48-22751100ee85 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:00.926737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1059/2083] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:50:00 2022] POST /identity/v3/projects => generated 401 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:50:00.927610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c92116-807c-48ba-8112-d2045f7d8fcc admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:00.929689 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a0c92116-807c-48ba-8112-d2045f7d8fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:00.997298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c92116-807c-48ba-8112-d2045f7d8fcc admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:01.001429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1025/2084] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:50:00 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:50:01.001807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82489ce7-56b6-4cb9-bf48-22751100ee85 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:01.006444 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:01.007088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.007369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.007647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] PATH_INFO: `/v3/projects/1a321d3be51341e6bc68b315a2781610` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:01.009455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a321d3be51341e6bc68b315a2781610)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:01.011621 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:01.071494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82489ce7-56b6-4cb9-bf48-22751100ee85 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:01.076251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec991ee1-a473-471b-9f8b-8016d5c684b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:01.076862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec991ee1-a473-471b-9f8b-8016d5c684b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.077154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec991ee1-a473-471b-9f8b-8016d5c684b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.077427 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec991ee1-a473-471b-9f8b-8016d5c684b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:01.078612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec991ee1-a473-471b-9f8b-8016d5c684b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:01.082375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec991ee1-a473-471b-9f8b-8016d5c684b5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:01.157826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a321d3be51341e6bc68b315a2781610', 'request_id': 'req-66af7624-5cea-4b5f-a642-5b25d044a00c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:01.165257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a321d3be51341e6bc68b315a2781610', 'request_id': 'req-66af7624-5cea-4b5f-a642-5b25d044a00c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:01.175333 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1026/2085] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:50:01 2022] POST /identity/v3/users => generated 307 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:01.175333 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec991ee1-a473-471b-9f8b-8016d5c684b5 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:01.175581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a321d3be51341e6bc68b315a2781610', 'request_id': 'req-66af7624-5cea-4b5f-a642-5b25d044a00c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:01.181075 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec991ee1-a473-471b-9f8b-8016d5c684b5 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:01.253199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ef24a-80e5-47b6-be1b-656f28dfecb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:01.253898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef24a-80e5-47b6-be1b-656f28dfecb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.254183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef24a-80e5-47b6-be1b-656f28dfecb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.254461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef24a-80e5-47b6-be1b-656f28dfecb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:01.255197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ef24a-80e5-47b6-be1b-656f28dfecb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:01.257177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ef24a-80e5-47b6-be1b-656f28dfecb7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:01.266722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] The token cache is being invalidate because project 1a321d3be51341e6bc68b315a2781610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:50:01.278219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:01.280969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:01.284420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1060/2086] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:50:00 2022] DELETE /identity/v3/projects/1a321d3be51341e6bc68b315a2781610 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:01.285099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:01.294456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:01.309853 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af7624-5cea-4b5f-a642-5b25d044a00c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:01.316095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a467b-1467-47ef-89a2-00d61c4e99fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:01.316803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a467b-1467-47ef-89a2-00d61c4e99fe admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.317299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a467b-1467-47ef-89a2-00d61c4e99fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.317672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a467b-1467-47ef-89a2-00d61c4e99fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:01.318491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a467b-1467-47ef-89a2-00d61c4e99fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:01.320237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a467b-1467-47ef-89a2-00d61c4e99fe admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:01.336826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1061/2087] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:01.338280 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a467b-1467-47ef-89a2-00d61c4e99fe admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:01.342558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c64a467b-1467-47ef-89a2-00d61c4e99fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:01.358156 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a467b-1467-47ef-89a2-00d61c4e99fe admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:01.360722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1027/2088] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:50:01 2022] POST /identity/v3/users => generated 282 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:01.365863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ef24a-80e5-47b6-be1b-656f28dfecb7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:01.366780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54f0fa-6818-4927-9432-8d0c3a2dda41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:01.367515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54f0fa-6818-4927-9432-8d0c3a2dda41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.367799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54f0fa-6818-4927-9432-8d0c3a2dda41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.368075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54f0fa-6818-4927-9432-8d0c3a2dda41 admin admin] PATH_INFO: `/v3/users/0a11abcc18764fe08a745e96122d5d4e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:01.369321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54f0fa-6818-4927-9432-8d0c3a2dda41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a11abcc18764fe08a745e96122d5d4e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:01.371349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54f0fa-6818-4927-9432-8d0c3a2dda41 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:01.374095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ef24a-80e5-47b6-be1b-656f28dfecb7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:01.379792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adbd0e-cdea-447d-9683-60f6793bafa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:01.380545 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adbd0e-cdea-447d-9683-60f6793bafa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.380831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adbd0e-cdea-447d-9683-60f6793bafa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.381106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adbd0e-cdea-447d-9683-60f6793bafa9 admin admin] PATH_INFO: `/v3/projects/699a12e4fee74854b6e25caedef3e8ef/users/5e698e2d15cf4cdca3f8883c5b81fadd/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:01.398156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adbd0e-cdea-447d-9683-60f6793bafa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=699a12e4fee74854b6e25caedef3e8ef, user_id=5e698e2d15cf4cdca3f8883c5b81fadd, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:01.401260 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adbd0e-cdea-447d-9683-60f6793bafa9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:01.544446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:01.550699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1028/2089] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:01 2022] PUT /identity/v3/projects/699a12e4fee74854b6e25caedef3e8ef/users/5e698e2d15cf4cdca3f8883c5b81fadd/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:01.551644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adbd0e-cdea-447d-9683-60f6793bafa9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:01.553218 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-56adbd0e-cdea-447d-9683-60f6793bafa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:01.596151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0e54f0fa-6818-4927-9432-8d0c3a2dda41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a11abcc18764fe08a745e96122d5d4e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:01.621428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adbd0e-cdea-447d-9683-60f6793bafa9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:01.626729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b2e82-6a1d-433d-9bae-10c1dc26cca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:01.627341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b2e82-6a1d-433d-9bae-10c1dc26cca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.627631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b2e82-6a1d-433d-9bae-10c1dc26cca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.627928 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b2e82-6a1d-433d-9bae-10c1dc26cca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:01.628574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b2e82-6a1d-433d-9bae-10c1dc26cca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:01.630214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b2e82-6a1d-433d-9bae-10c1dc26cca5 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:01.640220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1029/2090] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:01.641657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b2e82-6a1d-433d-9bae-10c1dc26cca5 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:01.651691 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b2e82-6a1d-433d-9bae-10c1dc26cca5 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:01.656482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500ce2d-92ba-4efa-86c4-55b4e91c0515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:01.657097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500ce2d-92ba-4efa-86c4-55b4e91c0515 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.657376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500ce2d-92ba-4efa-86c4-55b4e91c0515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.657745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500ce2d-92ba-4efa-86c4-55b4e91c0515 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:01.658428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500ce2d-92ba-4efa-86c4-55b4e91c0515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:01.660050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500ce2d-92ba-4efa-86c4-55b4e91c0515 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:01.670549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1030/2091] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:01.671372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500ce2d-92ba-4efa-86c4-55b4e91c0515 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:01.682351 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500ce2d-92ba-4efa-86c4-55b4e91c0515 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:01.687278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad40da-0555-46d7-83df-198cff583bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:01.688473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad40da-0555-46d7-83df-198cff583bbf admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.688749 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad40da-0555-46d7-83df-198cff583bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.689020 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad40da-0555-46d7-83df-198cff583bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:01.689706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad40da-0555-46d7-83df-198cff583bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:01.691290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad40da-0555-46d7-83df-198cff583bbf admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:01.702149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1031/2092] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:01.703058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad40da-0555-46d7-83df-198cff583bbf admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:01.713617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad40da-0555-46d7-83df-198cff583bbf admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:01.718519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2493b-2923-4a73-8a69-87bb3bfd700f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:01.719221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2493b-2923-4a73-8a69-87bb3bfd700f admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.719522 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2493b-2923-4a73-8a69-87bb3bfd700f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.719816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2493b-2923-4a73-8a69-87bb3bfd700f admin admin] PATH_INFO: `/v3/projects/699a12e4fee74854b6e25caedef3e8ef/users/5e698e2d15cf4cdca3f8883c5b81fadd/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:01.723539 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2493b-2923-4a73-8a69-87bb3bfd700f admin admin] RBAC: Authorizing `identity:create_grant(project_id=699a12e4fee74854b6e25caedef3e8ef, user_id=5e698e2d15cf4cdca3f8883c5b81fadd, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:01.726531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2493b-2923-4a73-8a69-87bb3bfd700f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:01.739962 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1032/2093] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:01 2022] PUT /identity/v3/projects/699a12e4fee74854b6e25caedef3e8ef/users/5e698e2d15cf4cdca3f8883c5b81fadd/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:01.742401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2493b-2923-4a73-8a69-87bb3bfd700f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:01.793337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0e54f0fa-6818-4927-9432-8d0c3a2dda41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a11abcc18764fe08a745e96122d5d4e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:01.794604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0e54f0fa-6818-4927-9432-8d0c3a2dda41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a11abcc18764fe08a745e96122d5d4e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:01.805591 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0e54f0fa-6818-4927-9432-8d0c3a2dda41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a11abcc18764fe08a745e96122d5d4e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:01.818407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2493b-2923-4a73-8a69-87bb3bfd700f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:01.824618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d08586-bf45-4bfe-a695-87ad180d6b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:01.825307 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d08586-bf45-4bfe-a695-87ad180d6b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.825653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d08586-bf45-4bfe-a695-87ad180d6b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.826007 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d08586-bf45-4bfe-a695-87ad180d6b43 admin admin] PATH_INFO: `/v3/projects/f6ca0e0cd370456d877f228ec50c4dc5/users/b23db9350127496fa138dad824635714/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:01.831039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0e54f0fa-6818-4927-9432-8d0c3a2dda41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a11abcc18764fe08a745e96122d5d4e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:01.840229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1062/2094] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:50:01 2022] DELETE /identity/v3/users/0a11abcc18764fe08a745e96122d5d4e => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:01.841623 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d08586-bf45-4bfe-a695-87ad180d6b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ca0e0cd370456d877f228ec50c4dc5, user_id=b23db9350127496fa138dad824635714, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:01.842153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f5d02-5211-4fbb-99d0-cfb130913a39 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.842568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f5d02-5211-4fbb-99d0-cfb130913a39 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.843007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f5d02-5211-4fbb-99d0-cfb130913a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:01.844411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d08586-bf45-4bfe-a695-87ad180d6b43 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:01.874365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-743f5d02-5211-4fbb-99d0-cfb130913a39 None None] MFA Rules not processed for user `5e698e2d15cf4cdca3f8883c5b81fadd`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:01.876133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-743f5d02-5211-4fbb-99d0-cfb130913a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:01.958886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1063/2095] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:01 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:01.964323 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1033/2096] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:01 2022] PUT /identity/v3/projects/f6ca0e0cd370456d877f228ec50c4dc5/users/b23db9350127496fa138dad824635714/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:01.970513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f5d02-5211-4fbb-99d0-cfb130913a39 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:01.972031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-743f5d02-5211-4fbb-99d0-cfb130913a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:01.977659 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cb489-8876-48ee-abf1-a1ccbd2acb45 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:01.978059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cb489-8876-48ee-abf1-a1ccbd2acb45 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:01.978334 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cb489-8876-48ee-abf1-a1ccbd2acb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:02.012699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-090cb489-8876-48ee-abf1-a1ccbd2acb45 None None] MFA Rules not processed for user `b23db9350127496fa138dad824635714`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:02.013954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-090cb489-8876-48ee-abf1-a1ccbd2acb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:02.038733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f5d02-5211-4fbb-99d0-cfb130913a39 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:02.084346 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1034/2097] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:50:02.095796 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c12d91-5591-4732-809b-5755744edfc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:02.096259 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cb489-8876-48ee-abf1-a1ccbd2acb45 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:02.096376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c12d91-5591-4732-809b-5755744edfc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:02.096635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c12d91-5591-4732-809b-5755744edfc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:02.096916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c12d91-5591-4732-809b-5755744edfc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:02.098955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-090cb489-8876-48ee-abf1-a1ccbd2acb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:02.112815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cb489-8876-48ee-abf1-a1ccbd2acb45 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:02.116961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1137b7c-fe9b-4bed-9ece-898996e6d4b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:02.117703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1137b7c-fe9b-4bed-9ece-898996e6d4b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:02.118064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1137b7c-fe9b-4bed-9ece-898996e6d4b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:02.118439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1137b7c-fe9b-4bed-9ece-898996e6d4b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:02.130297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1137b7c-fe9b-4bed-9ece-898996e6d4b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:02.132371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1137b7c-fe9b-4bed-9ece-898996e6d4b7 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:02.146428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1035/2098] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:50:02.176445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c12d91-5591-4732-809b-5755744edfc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:02.178963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c12d91-5591-4732-809b-5755744edfc7 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:02.193127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:02.196676 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1064/2099] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:01 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:50:02.375367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1137b7c-fe9b-4bed-9ece-898996e6d4b7 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:02.376822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c1137b7c-fe9b-4bed-9ece-898996e6d4b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:02.457785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1137b7c-fe9b-4bed-9ece-898996e6d4b7 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:02.462502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:02.463157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:02.463434 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:02.463734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] PATH_INFO: `/v3/projects/a02d743fcb964cbebff55822a729ea55` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:02.464916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a02d743fcb964cbebff55822a729ea55)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:02.466883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:02.546378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c12d91-5591-4732-809b-5755744edfc7 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:02.558872 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c12d91-5591-4732-809b-5755744edfc7 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:02.563731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802248c1-b84f-4654-b946-8043fa54c340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:02.564337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802248c1-b84f-4654-b946-8043fa54c340 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:02.564610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802248c1-b84f-4654-b946-8043fa54c340 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:02.564878 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802248c1-b84f-4654-b946-8043fa54c340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:02.567452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-802248c1-b84f-4654-b946-8043fa54c340 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:02.602438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a02d743fcb964cbebff55822a729ea55', 'request_id': 'req-2b105ca5-ea3f-4939-b9b6-27fc446904ab'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:02.610485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a02d743fcb964cbebff55822a729ea55', 'request_id': 'req-2b105ca5-ea3f-4939-b9b6-27fc446904ab'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:02.617612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a02d743fcb964cbebff55822a729ea55', 'request_id': 'req-2b105ca5-ea3f-4939-b9b6-27fc446904ab'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:02.643750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802248c1-b84f-4654-b946-8043fa54c340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:02.645203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802248c1-b84f-4654-b946-8043fa54c340 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:02.654155 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1065/2100] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:50:02.718670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] The token cache is being invalidate because project a02d743fcb964cbebff55822a729ea55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:50:02.719543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:02.719889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:02.725669 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1036/2101] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:50:02 2022] DELETE /identity/v3/projects/a02d743fcb964cbebff55822a729ea55 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:02.727036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802248c1-b84f-4654-b946-8043fa54c340 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:02.742422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802248c1-b84f-4654-b946-8043fa54c340 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:02.809784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee451b54-d512-495d-9a50-a41b1d38328a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:02.810425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee451b54-d512-495d-9a50-a41b1d38328a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:02.810706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee451b54-d512-495d-9a50-a41b1d38328a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:02.810980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee451b54-d512-495d-9a50-a41b1d38328a admin admin] PATH_INFO: `/v3/users/185b33480dbb4a099de834c8dad310f3` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:02.812305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee451b54-d512-495d-9a50-a41b1d38328a admin admin] RBAC: Authorizing `identity:delete_user(user_id=185b33480dbb4a099de834c8dad310f3)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:02.814362 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee451b54-d512-495d-9a50-a41b1d38328a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:02.845321 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:02.926671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:03.054595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ee451b54-d512-495d-9a50-a41b1d38328a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '185b33480dbb4a099de834c8dad310f3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:03.160709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ee451b54-d512-495d-9a50-a41b1d38328a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '185b33480dbb4a099de834c8dad310f3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:03.161342 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ee451b54-d512-495d-9a50-a41b1d38328a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '185b33480dbb4a099de834c8dad310f3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:03.172886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ee451b54-d512-495d-9a50-a41b1d38328a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '185b33480dbb4a099de834c8dad310f3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:03.186475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ee451b54-d512-495d-9a50-a41b1d38328a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '185b33480dbb4a099de834c8dad310f3'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:03.192610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1066/2102] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:50:02 2022] DELETE /identity/v3/users/185b33480dbb4a099de834c8dad310f3 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:03.805228 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:03.852711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:03.854493 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:03.869750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b105ca5-ea3f-4939-b9b6-27fc446904ab admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:03.936098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d865569-f583-48b1-a1a9-b87159ae5ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:03.936745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d865569-f583-48b1-a1a9-b87159ae5ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:03.937030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d865569-f583-48b1-a1a9-b87159ae5ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:03.937350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d865569-f583-48b1-a1a9-b87159ae5ce1 admin admin] PATH_INFO: `/v3/projects/6a8460f6cb7c451db702992d62de1b1e` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:03.939068 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d865569-f583-48b1-a1a9-b87159ae5ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a8460f6cb7c451db702992d62de1b1e)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:03.941043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d865569-f583-48b1-a1a9-b87159ae5ce1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:04.148923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8d865569-f583-48b1-a1a9-b87159ae5ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a8460f6cb7c451db702992d62de1b1e', 'request_id': 'req-8d865569-f583-48b1-a1a9-b87159ae5ce1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:04.155723 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8d865569-f583-48b1-a1a9-b87159ae5ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a8460f6cb7c451db702992d62de1b1e', 'request_id': 'req-8d865569-f583-48b1-a1a9-b87159ae5ce1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:04.162144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8d865569-f583-48b1-a1a9-b87159ae5ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a8460f6cb7c451db702992d62de1b1e', 'request_id': 'req-8d865569-f583-48b1-a1a9-b87159ae5ce1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:04.166638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:04.274039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:04.280146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8d865569-f583-48b1-a1a9-b87159ae5ce1 admin admin] The token cache is being invalidate because project 6a8460f6cb7c451db702992d62de1b1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:50:04.282344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8d865569-f583-48b1-a1a9-b87159ae5ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:04.282344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8d865569-f583-48b1-a1a9-b87159ae5ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:04.287509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1037/2103] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:50:03 2022] DELETE /identity/v3/projects/6a8460f6cb7c451db702992d62de1b1e => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:50:04.288427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c7806-bcb8-4c53-8308-0da515fb5c01 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:04.288724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c7806-bcb8-4c53-8308-0da515fb5c01 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:04.289013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c7806-bcb8-4c53-8308-0da515fb5c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:04.310982 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-618c7806-bcb8-4c53-8308-0da515fb5c01 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:04.312247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-618c7806-bcb8-4c53-8308-0da515fb5c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:04.373132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1067/2104] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:04.382442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226663f5-b925-4580-9c1f-fb080b23cf75 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:04.382727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226663f5-b925-4580-9c1f-fb080b23cf75 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:04.383033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226663f5-b925-4580-9c1f-fb080b23cf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:04.402969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-226663f5-b925-4580-9c1f-fb080b23cf75 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:04.404182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-226663f5-b925-4580-9c1f-fb080b23cf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:04.412896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1038/2105] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:04.418652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c7806-bcb8-4c53-8308-0da515fb5c01 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:04.430131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c7806-bcb8-4c53-8308-0da515fb5c01 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:04.435212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652bfc3-e0e8-42ec-a2f1-d0d587127bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:04.435894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652bfc3-e0e8-42ec-a2f1-d0d587127bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:04.436201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652bfc3-e0e8-42ec-a2f1-d0d587127bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:04.436477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652bfc3-e0e8-42ec-a2f1-d0d587127bf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:04.437168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652bfc3-e0e8-42ec-a2f1-d0d587127bf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:50:04.437659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652bfc3-e0e8-42ec-a2f1-d0d587127bf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:04.439391 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652bfc3-e0e8-42ec-a2f1-d0d587127bf8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:04.455230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1068/2106] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:50:04.457169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226663f5-b925-4580-9c1f-fb080b23cf75 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:04.469464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226663f5-b925-4580-9c1f-fb080b23cf75 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:04.472985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b342f3-3fd6-40c3-81d4-e83d96e42dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:04.473702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b342f3-3fd6-40c3-81d4-e83d96e42dcc admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:04.473980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b342f3-3fd6-40c3-81d4-e83d96e42dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:04.474252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b342f3-3fd6-40c3-81d4-e83d96e42dcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:04.474898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b342f3-3fd6-40c3-81d4-e83d96e42dcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:04.476850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b342f3-3fd6-40c3-81d4-e83d96e42dcc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:04.697564 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1039/2107] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:50:04 2022] POST /identity/v3/projects => generated 371 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:50:04.700751 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652bfc3-e0e8-42ec-a2f1-d0d587127bf8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:04.723599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652bfc3-e0e8-42ec-a2f1-d0d587127bf8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:04.795416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6ba94-f13f-46f1-99a1-f04e98a5e70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:04.796019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6ba94-f13f-46f1-99a1-f04e98a5e70a admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:04.796514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6ba94-f13f-46f1-99a1-f04e98a5e70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:04.796797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6ba94-f13f-46f1-99a1-f04e98a5e70a admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:04.797467 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6ba94-f13f-46f1-99a1-f04e98a5e70a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:04.801321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6ba94-f13f-46f1-99a1-f04e98a5e70a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:04.920222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:04.927216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1069/2108] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:50:04 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:04.931309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b342f3-3fd6-40c3-81d4-e83d96e42dcc admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:04.942298 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b342f3-3fd6-40c3-81d4-e83d96e42dcc admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:04.946884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47302e6f-68ff-423c-9820-03988d22fd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:04.947483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47302e6f-68ff-423c-9820-03988d22fd07 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:04.947766 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47302e6f-68ff-423c-9820-03988d22fd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:04.948043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47302e6f-68ff-423c-9820-03988d22fd07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:04.948690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47302e6f-68ff-423c-9820-03988d22fd07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:04.950346 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47302e6f-68ff-423c-9820-03988d22fd07 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:04.960772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1040/2109] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:50:04.971228 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6ba94-f13f-46f1-99a1-f04e98a5e70a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:04.981859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6ba94-f13f-46f1-99a1-f04e98a5e70a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:04.986873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62305ea7-21d5-4ab5-a41b-17b7ac76a6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:04.987488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62305ea7-21d5-4ab5-a41b-17b7ac76a6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:04.987821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62305ea7-21d5-4ab5-a41b-17b7ac76a6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:04.988104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62305ea7-21d5-4ab5-a41b-17b7ac76a6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:04.988754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62305ea7-21d5-4ab5-a41b-17b7ac76a6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:04.990452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62305ea7-21d5-4ab5-a41b-17b7ac76a6d6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:05.002330 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1070/2110] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:05.006653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47302e6f-68ff-423c-9820-03988d22fd07 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:05.018734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47302e6f-68ff-423c-9820-03988d22fd07 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:05.024693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5246684-6f95-491e-984f-89e51a0a56c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:05.025452 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5246684-6f95-491e-984f-89e51a0a56c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:05.025773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5246684-6f95-491e-984f-89e51a0a56c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:05.026124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5246684-6f95-491e-984f-89e51a0a56c9 admin admin] PATH_INFO: `/v3/projects/6efde3fd7658438cad16413a10503384/users/dc285681bd4843028f0fbbbd01ff4adb/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:05.041139 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5246684-6f95-491e-984f-89e51a0a56c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6efde3fd7658438cad16413a10503384, user_id=dc285681bd4843028f0fbbbd01ff4adb, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:05.043935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5246684-6f95-491e-984f-89e51a0a56c9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:05.159917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1041/2111] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:05 2022] PUT /identity/v3/projects/6efde3fd7658438cad16413a10503384/users/dc285681bd4843028f0fbbbd01ff4adb/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:50:05.165332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34e255-12ed-4908-9c55-0712750af294 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:05.165825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34e255-12ed-4908-9c55-0712750af294 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:05.169939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34e255-12ed-4908-9c55-0712750af294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:05.186898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1c34e255-12ed-4908-9c55-0712750af294 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:05.188641 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1c34e255-12ed-4908-9c55-0712750af294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:05.247581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1071/2112] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:05.261897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5246684-6f95-491e-984f-89e51a0a56c9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:05.264060 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d5246684-6f95-491e-984f-89e51a0a56c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:05.281081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:05.340853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5246684-6f95-491e-984f-89e51a0a56c9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:05.344824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd840ef-e8f9-4eb1-9989-1b523bd65f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:05.345433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd840ef-e8f9-4eb1-9989-1b523bd65f41 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:05.345828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd840ef-e8f9-4eb1-9989-1b523bd65f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:05.346066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd840ef-e8f9-4eb1-9989-1b523bd65f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:05.355495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd840ef-e8f9-4eb1-9989-1b523bd65f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:05.356929 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd840ef-e8f9-4eb1-9989-1b523bd65f41 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:05.367823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1042/2113] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:50:06.925567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:07.839205 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:08.946280 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34e255-12ed-4908-9c55-0712750af294 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:08.947893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1c34e255-12ed-4908-9c55-0712750af294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:08.960426 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34e255-12ed-4908-9c55-0712750af294 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:08.966247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:08.967068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:08.971434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:08.971434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] PATH_INFO: `/v3/users/e695fecba1f84058a9d6057e5a9a9095` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:08.971434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e695fecba1f84058a9d6057e5a9a9095)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:08.971697 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:09.403894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e695fecba1f84058a9d6057e5a9a9095'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:09.536023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e695fecba1f84058a9d6057e5a9a9095'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:09.537212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e695fecba1f84058a9d6057e5a9a9095'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:09.545814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e695fecba1f84058a9d6057e5a9a9095'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:09.561374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e695fecba1f84058a9d6057e5a9a9095'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:09.571405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1072/2114] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:50:08 2022] DELETE /identity/v3/users/e695fecba1f84058a9d6057e5a9a9095 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:09.575991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978e60d-154d-4e09-a60a-058a7dd4da05 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:09.576458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978e60d-154d-4e09-a60a-058a7dd4da05 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:09.576539 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978e60d-154d-4e09-a60a-058a7dd4da05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:09.603403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a978e60d-154d-4e09-a60a-058a7dd4da05 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:09.605045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a978e60d-154d-4e09-a60a-058a7dd4da05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:09.691756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1043/2115] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:09.702371 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:09.704470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:09.816870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adf15d-96c7-459f-9142-3c32b659851f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:09.828732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740a2d2-e680-4ec4-94c9-8b271f9e6360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:09.829258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740a2d2-e680-4ec4-94c9-8b271f9e6360 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:09.829790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740a2d2-e680-4ec4-94c9-8b271f9e6360 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:09.830088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740a2d2-e680-4ec4-94c9-8b271f9e6360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:09.850827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740a2d2-e680-4ec4-94c9-8b271f9e6360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:09.855385 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740a2d2-e680-4ec4-94c9-8b271f9e6360 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:09.917433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1073/2116] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:50:10.032969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:10.767242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978e60d-154d-4e09-a60a-058a7dd4da05 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:10.770862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a978e60d-154d-4e09-a60a-058a7dd4da05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:10.788603 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978e60d-154d-4e09-a60a-058a7dd4da05 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:10.799001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:10.817780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:10.818102 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:10.818393 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] PATH_INFO: `/v3/projects/931be0e8324348acaa6bbbf3b7481ff6` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:10.821023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] RBAC: Authorizing `identity:delete_project(project_id=931be0e8324348acaa6bbbf3b7481ff6)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:10.824824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:10.863662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:10.942474 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a361a3-e7b3-4ff2-a1a7-c60577d47a25 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:10.942768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a361a3-e7b3-4ff2-a1a7-c60577d47a25 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:10.943033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a361a3-e7b3-4ff2-a1a7-c60577d47a25 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:10.944394 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1074/2117] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:50:10.984872 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128d541-c941-45e4-b369-3499744ec22d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:10.985157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128d541-c941-45e4-b369-3499744ec22d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:10.985431 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128d541-c941-45e4-b369-3499744ec22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:11.016623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4128d541-c941-45e4-b369-3499744ec22d None None] MFA Rules not processed for user `5e0aa6b215f646c280af245a89ccf849`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:11.018197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4128d541-c941-45e4-b369-3499744ec22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:11.042906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '931be0e8324348acaa6bbbf3b7481ff6', 'request_id': 'req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.051493 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '931be0e8324348acaa6bbbf3b7481ff6', 'request_id': 'req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.070789 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '931be0e8324348acaa6bbbf3b7481ff6', 'request_id': 'req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.121351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1075/2118] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:50:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:11.134616 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128d541-c941-45e4-b369-3499744ec22d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:11.136062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4128d541-c941-45e4-b369-3499744ec22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:11.152901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128d541-c941-45e4-b369-3499744ec22d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:11.158960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56afe43-32f7-408f-8367-dcd4c98a6b32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0aa6b215f646c280af245a89ccf849', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:11.159604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56afe43-32f7-408f-8367-dcd4c98a6b32 service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:11.159888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56afe43-32f7-408f-8367-dcd4c98a6b32 service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:11.160163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56afe43-32f7-408f-8367-dcd4c98a6b32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:11.168971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56afe43-32f7-408f-8367-dcd4c98a6b32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:11.170622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56afe43-32f7-408f-8367-dcd4c98a6b32 service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:11.183066 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1076/2119] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:50:11.190074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] The token cache is being invalidate because project 931be0e8324348acaa6bbbf3b7481ff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:50:11.190380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.191047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.192922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1044/2120] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:50:10 2022] DELETE /identity/v3/projects/931be0e8324348acaa6bbbf3b7481ff6 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:11.196745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56afe43-32f7-408f-8367-dcd4c98a6b32 service placement] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:11.201872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:11.203432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:11.299840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56afe43-32f7-408f-8367-dcd4c98a6b32 service placement] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:11.305747 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb316e-54eb-4a6e-8a4f-f22dba1df52b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:11.307685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:11.308329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:11.308611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:11.308952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] PATH_INFO: `/v3/users/5e698e2d15cf4cdca3f8883c5b81fadd` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:11.310054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e698e2d15cf4cdca3f8883c5b81fadd)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:11.310442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:11.311096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:11.311372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:11.311640 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] PATH_INFO: `/v3/users/7fde48fb289b4c7591658e27ce5ff55e` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:11.312048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:11.312872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fde48fb289b4c7591658e27ce5ff55e)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:11.314792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:11.502692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:11.519269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e698e2d15cf4cdca3f8883c5b81fadd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.520954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fde48fb289b4c7591658e27ce5ff55e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.622498 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e698e2d15cf4cdca3f8883c5b81fadd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.623198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e698e2d15cf4cdca3f8883c5b81fadd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.632113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e698e2d15cf4cdca3f8883c5b81fadd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.646127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e698e2d15cf4cdca3f8883c5b81fadd'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.667399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1077/2121] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:50:11 2022] DELETE /identity/v3/users/5e698e2d15cf4cdca3f8883c5b81fadd => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:50:11.684318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fde48fb289b4c7591658e27ce5ff55e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.685050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fde48fb289b4c7591658e27ce5ff55e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.701140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fde48fb289b4c7591658e27ce5ff55e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.716650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fde48fb289b4c7591658e27ce5ff55e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:11.726774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1045/2122] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:50:11 2022] DELETE /identity/v3/users/7fde48fb289b4c7591658e27ce5ff55e => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:50:12.222093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:12.223769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:12.280451 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:12.280451 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:12.317344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29316962-c633-4e15-9b7f-665f0b08eab3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:12.379563 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:12.386151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11918da-135a-4d14-ab5e-e002bee8392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:12.387002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11918da-135a-4d14-ab5e-e002bee8392d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:12.387428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11918da-135a-4d14-ab5e-e002bee8392d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:12.388670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11918da-135a-4d14-ab5e-e002bee8392d admin admin] PATH_INFO: `/v3/projects/539b956efa1b47799514fbdcac7e250c` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:12.390158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11918da-135a-4d14-ab5e-e002bee8392d admin admin] RBAC: Authorizing `identity:delete_project(project_id=539b956efa1b47799514fbdcac7e250c)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:12.390753 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b61ef-d308-4b96-bc34-c49c209a5ea2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:12.392366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11918da-135a-4d14-ab5e-e002bee8392d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:12.398743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728e44c-e9e5-4c59-aceb-d939ed325f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:12.399477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728e44c-e9e5-4c59-aceb-d939ed325f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:12.399830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728e44c-e9e5-4c59-aceb-d939ed325f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:12.400172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728e44c-e9e5-4c59-aceb-d939ed325f11 admin admin] PATH_INFO: `/v3/projects/699a12e4fee74854b6e25caedef3e8ef` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:12.401400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728e44c-e9e5-4c59-aceb-d939ed325f11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=699a12e4fee74854b6e25caedef3e8ef)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:12.404707 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728e44c-e9e5-4c59-aceb-d939ed325f11 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:12.595220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c11918da-135a-4d14-ab5e-e002bee8392d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '539b956efa1b47799514fbdcac7e250c', 'request_id': 'req-c11918da-135a-4d14-ab5e-e002bee8392d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:12.602244 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c11918da-135a-4d14-ab5e-e002bee8392d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '539b956efa1b47799514fbdcac7e250c', 'request_id': 'req-c11918da-135a-4d14-ab5e-e002bee8392d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:12.610655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c11918da-135a-4d14-ab5e-e002bee8392d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '539b956efa1b47799514fbdcac7e250c', 'request_id': 'req-c11918da-135a-4d14-ab5e-e002bee8392d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:12.641626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6728e44c-e9e5-4c59-aceb-d939ed325f11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '699a12e4fee74854b6e25caedef3e8ef', 'request_id': 'req-6728e44c-e9e5-4c59-aceb-d939ed325f11'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:12.648501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6728e44c-e9e5-4c59-aceb-d939ed325f11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '699a12e4fee74854b6e25caedef3e8ef', 'request_id': 'req-6728e44c-e9e5-4c59-aceb-d939ed325f11'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:12.654799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6728e44c-e9e5-4c59-aceb-d939ed325f11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '699a12e4fee74854b6e25caedef3e8ef', 'request_id': 'req-6728e44c-e9e5-4c59-aceb-d939ed325f11'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:12.763472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c11918da-135a-4d14-ab5e-e002bee8392d admin admin] The token cache is being invalidate because project 539b956efa1b47799514fbdcac7e250c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:50:12.763660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c11918da-135a-4d14-ab5e-e002bee8392d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:12.764348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c11918da-135a-4d14-ab5e-e002bee8392d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:12.766255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1078/2123] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:50:12 2022] DELETE /identity/v3/projects/539b956efa1b47799514fbdcac7e250c => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:12.771085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c3e25-f3de-4d73-831d-9ed4f8dd2d61 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:12.771487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c3e25-f3de-4d73-831d-9ed4f8dd2d61 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:12.771867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c3e25-f3de-4d73-831d-9ed4f8dd2d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:12.800195 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-099c3e25-f3de-4d73-831d-9ed4f8dd2d61 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:12.801461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-099c3e25-f3de-4d73-831d-9ed4f8dd2d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:12.867217 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6728e44c-e9e5-4c59-aceb-d939ed325f11 admin admin] The token cache is being invalidate because project 699a12e4fee74854b6e25caedef3e8ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:50:12.867217 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6728e44c-e9e5-4c59-aceb-d939ed325f11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:12.874232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6728e44c-e9e5-4c59-aceb-d939ed325f11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:12.881249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1046/2124] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:50:12 2022] DELETE /identity/v3/projects/699a12e4fee74854b6e25caedef3e8ef => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:50:12.884310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1393700-a6e6-4371-84df-b26a2ff9c700 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:12.884598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1393700-a6e6-4371-84df-b26a2ff9c700 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:12.884878 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1393700-a6e6-4371-84df-b26a2ff9c700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:12.910205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c1393700-a6e6-4371-84df-b26a2ff9c700 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:12.919823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c1393700-a6e6-4371-84df-b26a2ff9c700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:12.927076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1079/2125] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:12.934911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab4ea0-c0d7-4a54-92c9-ef8b88bd5d35 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:12.935201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab4ea0-c0d7-4a54-92c9-ef8b88bd5d35 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:12.935476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab4ea0-c0d7-4a54-92c9-ef8b88bd5d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:12.956939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2aab4ea0-c0d7-4a54-92c9-ef8b88bd5d35 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:12.958340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2aab4ea0-c0d7-4a54-92c9-ef8b88bd5d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:12.966628 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1080/2126] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:12.971882 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab4ea0-c0d7-4a54-92c9-ef8b88bd5d35 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:12.982279 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab4ea0-c0d7-4a54-92c9-ef8b88bd5d35 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:12.985972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1047/2127] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:50:12.993027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27919463-b96c-491f-ac66-8f5bdb9bdae5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:12.993761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27919463-b96c-491f-ac66-8f5bdb9bdae5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:12.993880 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3e439-a94b-4214-9cbd-86237e176088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:12.994058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27919463-b96c-491f-ac66-8f5bdb9bdae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:12.995735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3e439-a94b-4214-9cbd-86237e176088 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:12.996194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3e439-a94b-4214-9cbd-86237e176088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:12.996476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3e439-a94b-4214-9cbd-86237e176088 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:12.998876 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3e439-a94b-4214-9cbd-86237e176088 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:50:13.000100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3e439-a94b-4214-9cbd-86237e176088 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.001023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3e439-a94b-4214-9cbd-86237e176088 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.022252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-27919463-b96c-491f-ac66-8f5bdb9bdae5 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:13.023954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-27919463-b96c-491f-ac66-8f5bdb9bdae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:13.032391 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1081/2128] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:50:13.038850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1048/2129] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:13.044490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27919463-b96c-491f-ac66-8f5bdb9bdae5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.046698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3e439-a94b-4214-9cbd-86237e176088 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.057410 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27919463-b96c-491f-ac66-8f5bdb9bdae5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.062873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a7c52-639f-42b9-a606-0a64729b9d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.063588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a7c52-639f-42b9-a606-0a64729b9d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.063936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a7c52-639f-42b9-a606-0a64729b9d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.064277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a7c52-639f-42b9-a606-0a64729b9d93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.065157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a7c52-639f-42b9-a606-0a64729b9d93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:50:13.065549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a7c52-639f-42b9-a606-0a64729b9d93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.069579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a7c52-639f-42b9-a606-0a64729b9d93 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.072043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3e439-a94b-4214-9cbd-86237e176088 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.081104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1049/2130] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:50:13.087106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a7c52-639f-42b9-a606-0a64729b9d93 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.089594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9694bc-c9af-41ec-8885-08589970abd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.090237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9694bc-c9af-41ec-8885-08589970abd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.090520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9694bc-c9af-41ec-8885-08589970abd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.090796 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9694bc-c9af-41ec-8885-08589970abd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.091621 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9694bc-c9af-41ec-8885-08589970abd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.095410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9694bc-c9af-41ec-8885-08589970abd9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.101140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a7c52-639f-42b9-a606-0a64729b9d93 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.106801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ee545-7c6f-4860-8bb1-696a122709ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.107634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ee545-7c6f-4860-8bb1-696a122709ee admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.108513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ee545-7c6f-4860-8bb1-696a122709ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.109090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ee545-7c6f-4860-8bb1-696a122709ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.110483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ee545-7c6f-4860-8bb1-696a122709ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.113025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ee545-7c6f-4860-8bb1-696a122709ee admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.269782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1082/2131] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:50:13 2022] POST /identity/v3/projects => generated 385 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:13.271382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9694bc-c9af-41ec-8885-08589970abd9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.277005 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1050/2132] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:50:13 2022] POST /identity/v3/projects => generated 407 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:13.278172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ee545-7c6f-4860-8bb1-696a122709ee admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.379173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ee545-7c6f-4860-8bb1-696a122709ee admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.388280 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfbb3f-3033-440a-9ffd-65fb6f1975d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.388920 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfbb3f-3033-440a-9ffd-65fb6f1975d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.389219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfbb3f-3033-440a-9ffd-65fb6f1975d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.389483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfbb3f-3033-440a-9ffd-65fb6f1975d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.390755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dfbb3f-3033-440a-9ffd-65fb6f1975d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.400220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dfbb3f-3033-440a-9ffd-65fb6f1975d2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.412434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9694bc-c9af-41ec-8885-08589970abd9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.420957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8eb936-75ab-478c-afd0-3b8f55be991e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.421645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eb936-75ab-478c-afd0-3b8f55be991e admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.421931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eb936-75ab-478c-afd0-3b8f55be991e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.422405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eb936-75ab-478c-afd0-3b8f55be991e admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.423567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8eb936-75ab-478c-afd0-3b8f55be991e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.426352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8eb936-75ab-478c-afd0-3b8f55be991e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.525221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1083/2133] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:50:13 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:13.528370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1051/2134] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:50:13 2022] POST /identity/v3/users => generated 304 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:13.529292 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8eb936-75ab-478c-afd0-3b8f55be991e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.532161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfbb3f-3033-440a-9ffd-65fb6f1975d2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.542472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8eb936-75ab-478c-afd0-3b8f55be991e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.543423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfbb3f-3033-440a-9ffd-65fb6f1975d2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.548683 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b5ba6-d32e-4167-afe8-52ffde0fb7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.555365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d1008-24df-4039-b7b1-e86243dac046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.555623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b5ba6-d32e-4167-afe8-52ffde0fb7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.555623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b5ba6-d32e-4167-afe8-52ffde0fb7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.555778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d1008-24df-4039-b7b1-e86243dac046 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.555854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b5ba6-d32e-4167-afe8-52ffde0fb7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.555923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d1008-24df-4039-b7b1-e86243dac046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.555923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d1008-24df-4039-b7b1-e86243dac046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.556046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b5ba6-d32e-4167-afe8-52ffde0fb7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.556155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d1008-24df-4039-b7b1-e86243dac046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.556229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b5ba6-d32e-4167-afe8-52ffde0fb7b6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.556309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d1008-24df-4039-b7b1-e86243dac046 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.569764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1084/2135] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:50:13.570874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b5ba6-d32e-4167-afe8-52ffde0fb7b6 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.572884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1052/2136] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:13.573945 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d1008-24df-4039-b7b1-e86243dac046 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.585957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b5ba6-d32e-4167-afe8-52ffde0fb7b6 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.586510 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d1008-24df-4039-b7b1-e86243dac046 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.591677 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53fdee-5018-4b36-a5f5-a1c29e9e42af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.591814 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53fdee-5018-4b36-a5f5-a1c29e9e42af admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.592106 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53fdee-5018-4b36-a5f5-a1c29e9e42af admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.592381 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53fdee-5018-4b36-a5f5-a1c29e9e42af admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.593200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53fdee-5018-4b36-a5f5-a1c29e9e42af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.593775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c1c2d-6b27-4886-9196-d19ab9498cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.594841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53fdee-5018-4b36-a5f5-a1c29e9e42af admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.595169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c1c2d-6b27-4886-9196-d19ab9498cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.595531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c1c2d-6b27-4886-9196-d19ab9498cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.595914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c1c2d-6b27-4886-9196-d19ab9498cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.597408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c1c2d-6b27-4886-9196-d19ab9498cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.599339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c1c2d-6b27-4886-9196-d19ab9498cc9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.606560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1053/2137] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:13.610900 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53fdee-5018-4b36-a5f5-a1c29e9e42af admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.617107 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1085/2138] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:13.620872 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c1c2d-6b27-4886-9196-d19ab9498cc9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.629284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53fdee-5018-4b36-a5f5-a1c29e9e42af admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.632348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c1c2d-6b27-4886-9196-d19ab9498cc9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.634630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5e8ac-6d1b-4ec8-b5bd-611229f986ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.635295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5e8ac-6d1b-4ec8-b5bd-611229f986ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.635577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5e8ac-6d1b-4ec8-b5bd-611229f986ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.635855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5e8ac-6d1b-4ec8-b5bd-611229f986ac admin admin] PATH_INFO: `/v3/projects/4a745db383cb452187f7c82b9977c09b/users/00491255f4b44125b6dc9f816c9bbb58/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.641954 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1ce9d-f846-4d49-9ef8-46f7dc1cabf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.642837 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1ce9d-f846-4d49-9ef8-46f7dc1cabf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.643451 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1ce9d-f846-4d49-9ef8-46f7dc1cabf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.644531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1ce9d-f846-4d49-9ef8-46f7dc1cabf2 admin admin] PATH_INFO: `/v3/projects/f5372d813a84489ebb1d95a56a24b609/users/c5d80285241948a8859fcd9e1b516da7/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.657700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5e8ac-6d1b-4ec8-b5bd-611229f986ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a745db383cb452187f7c82b9977c09b, user_id=00491255f4b44125b6dc9f816c9bbb58, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.660863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5e8ac-6d1b-4ec8-b5bd-611229f986ac admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.665905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1ce9d-f846-4d49-9ef8-46f7dc1cabf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5372d813a84489ebb1d95a56a24b609, user_id=c5d80285241948a8859fcd9e1b516da7, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.669690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1ce9d-f846-4d49-9ef8-46f7dc1cabf2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.755096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1054/2139] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:13 2022] PUT /identity/v3/projects/4a745db383cb452187f7c82b9977c09b/users/00491255f4b44125b6dc9f816c9bbb58/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:50:13.758536 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1086/2140] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:13 2022] PUT /identity/v3/projects/f5372d813a84489ebb1d95a56a24b609/users/c5d80285241948a8859fcd9e1b516da7/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:50:13.767683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de7056-f316-48a4-a150-6d0ef8e7fd23 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.768061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de7056-f316-48a4-a150-6d0ef8e7fd23 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.768403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de7056-f316-48a4-a150-6d0ef8e7fd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.770271 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9754e-11d0-4213-a65e-af8c67531fdf None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.770638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9754e-11d0-4213-a65e-af8c67531fdf None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.770981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9754e-11d0-4213-a65e-af8c67531fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.790854 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-07de7056-f316-48a4-a150-6d0ef8e7fd23 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:13.792072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-07de7056-f316-48a4-a150-6d0ef8e7fd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:13.802833 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-19f9754e-11d0-4213-a65e-af8c67531fdf None None] MFA Rules not processed for user `c5d80285241948a8859fcd9e1b516da7`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:13.804086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-19f9754e-11d0-4213-a65e-af8c67531fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:13.861299 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1055/2141] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:13.871472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de7056-f316-48a4-a150-6d0ef8e7fd23 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.874300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-07de7056-f316-48a4-a150-6d0ef8e7fd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:13.887288 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1087/2142] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:13 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:50:13.898009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9754e-11d0-4213-a65e-af8c67531fdf None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:13.929372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9754e-11d0-4213-a65e-af8c67531fdf None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.938284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3241b3-9d06-4d6a-b929-bece870b6624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.938919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3241b3-9d06-4d6a-b929-bece870b6624 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.939202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3241b3-9d06-4d6a-b929-bece870b6624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.939496 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3241b3-9d06-4d6a-b929-bece870b6624 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:13.957548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3241b3-9d06-4d6a-b929-bece870b6624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:13.961714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3241b3-9d06-4d6a-b929-bece870b6624 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:13.967623 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de7056-f316-48a4-a150-6d0ef8e7fd23 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:13.986663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e703f-92eb-4079-aff5-da8bad7c2b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:13.988386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e703f-92eb-4079-aff5-da8bad7c2b92 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:13.989176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e703f-92eb-4079-aff5-da8bad7c2b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:13.989655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e703f-92eb-4079-aff5-da8bad7c2b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:14.016842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e703f-92eb-4079-aff5-da8bad7c2b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:14.018589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e703f-92eb-4079-aff5-da8bad7c2b92 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:14.040256 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1056/2143] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:50:14.126666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1088/2144] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:50:13 2022] POST /identity/v3/projects => generated 409 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:14.130620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e703f-92eb-4079-aff5-da8bad7c2b92 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:14.147253 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e703f-92eb-4079-aff5-da8bad7c2b92 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:14.243498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89bfd9-8d41-469c-8983-a6d275bf968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:14.244127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89bfd9-8d41-469c-8983-a6d275bf968a admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:14.244410 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89bfd9-8d41-469c-8983-a6d275bf968a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:14.244686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89bfd9-8d41-469c-8983-a6d275bf968a admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:14.255322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89bfd9-8d41-469c-8983-a6d275bf968a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:14.255322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89bfd9-8d41-469c-8983-a6d275bf968a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:14.395616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1057/2145] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:50:14 2022] POST /identity/v3/users => generated 311 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:50:14.399679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3241b3-9d06-4d6a-b929-bece870b6624 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:14.415819 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3241b3-9d06-4d6a-b929-bece870b6624 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:14.436495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af152e5-e0fe-4277-ab80-4167695fdb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:14.439673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af152e5-e0fe-4277-ab80-4167695fdb1c admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:14.439673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af152e5-e0fe-4277-ab80-4167695fdb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:14.439673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af152e5-e0fe-4277-ab80-4167695fdb1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:14.439673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af152e5-e0fe-4277-ab80-4167695fdb1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:14.441162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af152e5-e0fe-4277-ab80-4167695fdb1c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:14.455491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1089/2146] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:14.460020 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89bfd9-8d41-469c-8983-a6d275bf968a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:14.475761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7523b07-ebcd-444d-97e6-3acb38ae5f44 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:14.476328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7523b07-ebcd-444d-97e6-3acb38ae5f44 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:14.477030 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7523b07-ebcd-444d-97e6-3acb38ae5f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:14.483724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89bfd9-8d41-469c-8983-a6d275bf968a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:14.502496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b6ca6-2373-40bf-a74f-8b34feab29ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:14.503188 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b6ca6-2373-40bf-a74f-8b34feab29ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:14.503711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b6ca6-2373-40bf-a74f-8b34feab29ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:14.504008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b6ca6-2373-40bf-a74f-8b34feab29ad admin admin] PATH_INFO: `/v3/projects/0fef2b8b86b341ad8904e5668819b8aa/users/b3f37a8ef4d64bdd80a17bd43ad3f81a/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:14.508109 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:14.524592 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b6ca6-2373-40bf-a74f-8b34feab29ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fef2b8b86b341ad8904e5668819b8aa, user_id=b3f37a8ef4d64bdd80a17bd43ad3f81a, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:14.528588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b6ca6-2373-40bf-a74f-8b34feab29ad admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:14.531528 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c7523b07-ebcd-444d-97e6-3acb38ae5f44 None None] MFA Rules not processed for user `dc285681bd4843028f0fbbbd01ff4adb`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:14.533342 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c7523b07-ebcd-444d-97e6-3acb38ae5f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:14.625757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1090/2147] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:50:14.628817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7523b07-ebcd-444d-97e6-3acb38ae5f44 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:14.630366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c7523b07-ebcd-444d-97e6-3acb38ae5f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:14.650667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7523b07-ebcd-444d-97e6-3acb38ae5f44 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:14.655731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f29be-4bbe-47b7-b618-beb0c3a248ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:14.656366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f29be-4bbe-47b7-b618-beb0c3a248ef admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:14.656648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f29be-4bbe-47b7-b618-beb0c3a248ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:14.657010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f29be-4bbe-47b7-b618-beb0c3a248ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:14.657714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f29be-4bbe-47b7-b618-beb0c3a248ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:14.659632 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f29be-4bbe-47b7-b618-beb0c3a248ef admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:14.676863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1058/2148] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:14 2022] PUT /identity/v3/projects/0fef2b8b86b341ad8904e5668819b8aa/users/b3f37a8ef4d64bdd80a17bd43ad3f81a/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:14.678389 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b6ca6-2373-40bf-a74f-8b34feab29ad admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:14.743144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b6ca6-2373-40bf-a74f-8b34feab29ad admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:14.749235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9e37f-74b1-4879-9eb2-7f100191af57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:14.750506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9e37f-74b1-4879-9eb2-7f100191af57 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:14.750776 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9e37f-74b1-4879-9eb2-7f100191af57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:14.751050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9e37f-74b1-4879-9eb2-7f100191af57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:14.751817 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9e37f-74b1-4879-9eb2-7f100191af57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:14.753432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9e37f-74b1-4879-9eb2-7f100191af57 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:14.764270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1059/2149] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:14.768159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9e37f-74b1-4879-9eb2-7f100191af57 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:14.779185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9e37f-74b1-4879-9eb2-7f100191af57 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:14.784561 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b2471-0be7-46cc-9d56-ddb27d14366a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:14.785226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b2471-0be7-46cc-9d56-ddb27d14366a admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:14.785507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b2471-0be7-46cc-9d56-ddb27d14366a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:14.785825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b2471-0be7-46cc-9d56-ddb27d14366a admin admin] PATH_INFO: `/v3/projects/0fef2b8b86b341ad8904e5668819b8aa/users/b3f37a8ef4d64bdd80a17bd43ad3f81a/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:14.787760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b2471-0be7-46cc-9d56-ddb27d14366a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fef2b8b86b341ad8904e5668819b8aa, user_id=b3f37a8ef4d64bdd80a17bd43ad3f81a, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:14.790850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b2471-0be7-46cc-9d56-ddb27d14366a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:14.803865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1060/2150] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:14 2022] PUT /identity/v3/projects/0fef2b8b86b341ad8904e5668819b8aa/users/b3f37a8ef4d64bdd80a17bd43ad3f81a/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:50:14.816741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbda30-43da-4f81-8e56-ebd4ff524654 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:14.817029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbda30-43da-4f81-8e56-ebd4ff524654 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:14.817304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbda30-43da-4f81-8e56-ebd4ff524654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:14.846454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5acbda30-43da-4f81-8e56-ebd4ff524654 None None] MFA Rules not processed for user `b3f37a8ef4d64bdd80a17bd43ad3f81a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:14.847661 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5acbda30-43da-4f81-8e56-ebd4ff524654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:14.849332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1091/2151] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:50:14 2022] POST /identity/v3/projects => generated 371 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:14.852854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f29be-4bbe-47b7-b618-beb0c3a248ef admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:14.916902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f29be-4bbe-47b7-b618-beb0c3a248ef admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:14.922553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8628f84-2b2b-4442-bfd8-5a652e7df61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:14.923132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8628f84-2b2b-4442-bfd8-5a652e7df61f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:14.923414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8628f84-2b2b-4442-bfd8-5a652e7df61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:14.923715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8628f84-2b2b-4442-bfd8-5a652e7df61f admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:14.924339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8628f84-2b2b-4442-bfd8-5a652e7df61f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:14.926591 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8628f84-2b2b-4442-bfd8-5a652e7df61f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:14.932022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1061/2152] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:14 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:14.942925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbda30-43da-4f81-8e56-ebd4ff524654 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:14.944775 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5acbda30-43da-4f81-8e56-ebd4ff524654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:15.016684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1092/2153] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:50:14 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:15.017839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbda30-43da-4f81-8e56-ebd4ff524654 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:15.019647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8628f84-2b2b-4442-bfd8-5a652e7df61f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:15.022604 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d72117-90f1-4628-9735-75ba1dd62e93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:15.023305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d72117-90f1-4628-9735-75ba1dd62e93 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:15.023659 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d72117-90f1-4628-9735-75ba1dd62e93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:15.024006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d72117-90f1-4628-9735-75ba1dd62e93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:15.034512 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8628f84-2b2b-4442-bfd8-5a652e7df61f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:15.036789 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d72117-90f1-4628-9735-75ba1dd62e93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:15.038403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d72117-90f1-4628-9735-75ba1dd62e93 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:15.038740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d0a4a-b570-45b7-af67-c4b788619e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:15.039367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d0a4a-b570-45b7-af67-c4b788619e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:15.039650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d0a4a-b570-45b7-af67-c4b788619e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:15.039928 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d0a4a-b570-45b7-af67-c4b788619e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:15.040591 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d0a4a-b570-45b7-af67-c4b788619e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:15.042240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d0a4a-b570-45b7-af67-c4b788619e3d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:15.052647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1062/2154] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:14 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:50:15.063202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1093/2155] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:15.065144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d72117-90f1-4628-9735-75ba1dd62e93 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:15.075292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d72117-90f1-4628-9735-75ba1dd62e93 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:15.082403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b0f04-b1f9-4ee4-a0fe-1fdfce5867c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:15.083091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b0f04-b1f9-4ee4-a0fe-1fdfce5867c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:15.086739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b0f04-b1f9-4ee4-a0fe-1fdfce5867c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:15.086739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b0f04-b1f9-4ee4-a0fe-1fdfce5867c3 admin admin] PATH_INFO: `/v3/projects/4d6e861a742c41c488ab1a4060f63fc9/users/4d941fd5e5fd4eb281fc92a71c32d6f9/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:15.102269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b0f04-b1f9-4ee4-a0fe-1fdfce5867c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6e861a742c41c488ab1a4060f63fc9, user_id=4d941fd5e5fd4eb281fc92a71c32d6f9, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:15.105920 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b0f04-b1f9-4ee4-a0fe-1fdfce5867c3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:15.194609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1063/2156] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:15 2022] PUT /identity/v3/projects/4d6e861a742c41c488ab1a4060f63fc9/users/4d941fd5e5fd4eb281fc92a71c32d6f9/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:15.198710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d0a4a-b570-45b7-af67-c4b788619e3d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:15.210290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d0a4a-b570-45b7-af67-c4b788619e3d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:15.297676 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab5541-cdfa-4417-a935-bc781935ed1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:15.298314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab5541-cdfa-4417-a935-bc781935ed1d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:15.298635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab5541-cdfa-4417-a935-bc781935ed1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:15.299732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab5541-cdfa-4417-a935-bc781935ed1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:15.300583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab5541-cdfa-4417-a935-bc781935ed1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:15.302237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab5541-cdfa-4417-a935-bc781935ed1d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:15.317881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1094/2157] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:15.326839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b0f04-b1f9-4ee4-a0fe-1fdfce5867c3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:15.338456 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b0f04-b1f9-4ee4-a0fe-1fdfce5867c3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:15.343984 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8711b7-9556-4f53-94d6-08b6888f652f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:15.344649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8711b7-9556-4f53-94d6-08b6888f652f admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:15.344930 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8711b7-9556-4f53-94d6-08b6888f652f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:15.345258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8711b7-9556-4f53-94d6-08b6888f652f admin admin] PATH_INFO: `/v3/projects/4d6e861a742c41c488ab1a4060f63fc9/users/4d941fd5e5fd4eb281fc92a71c32d6f9/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:15.347306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8711b7-9556-4f53-94d6-08b6888f652f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6e861a742c41c488ab1a4060f63fc9, user_id=4d941fd5e5fd4eb281fc92a71c32d6f9, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:15.352377 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8711b7-9556-4f53-94d6-08b6888f652f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:15.361932 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1064/2158] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:15 2022] PUT /identity/v3/projects/4d6e861a742c41c488ab1a4060f63fc9/users/4d941fd5e5fd4eb281fc92a71c32d6f9/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:15.529194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab5541-cdfa-4417-a935-bc781935ed1d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:15.600860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab5541-cdfa-4417-a935-bc781935ed1d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:15.653301 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:15.680287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81182e2-b02a-491c-8a32-0605c65d99cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:15.681823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81182e2-b02a-491c-8a32-0605c65d99cb service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:15.682275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81182e2-b02a-491c-8a32-0605c65d99cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:15.682832 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81182e2-b02a-491c-8a32-0605c65d99cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:15.793707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81182e2-b02a-491c-8a32-0605c65d99cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:15.797007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81182e2-b02a-491c-8a32-0605c65d99cb service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:15.847281 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1095/2159] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:15 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:50:18.730749 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:19.873627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:20.770671 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:22.187732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:22.611165 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f80ae8-798b-4408-a01b-4f08a17e13de None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:22.611674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f80ae8-798b-4408-a01b-4f08a17e13de None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:22.611789 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f80ae8-798b-4408-a01b-4f08a17e13de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:22.650916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e0f80ae8-798b-4408-a01b-4f08a17e13de None None] MFA Rules not processed for user `00491255f4b44125b6dc9f816c9bbb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:22.652531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e0f80ae8-798b-4408-a01b-4f08a17e13de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:22.769818 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1065/2160] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:50:22.786605 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81182e2-b02a-491c-8a32-0605c65d99cb service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:22.788237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c81182e2-b02a-491c-8a32-0605c65d99cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:22.928787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81182e2-b02a-491c-8a32-0605c65d99cb service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:22.935918 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833ddeb-900c-4078-83cc-b87fa63c9732 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:22.937335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833ddeb-900c-4078-83cc-b87fa63c9732 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:22.942165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833ddeb-900c-4078-83cc-b87fa63c9732 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:22.942165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833ddeb-900c-4078-83cc-b87fa63c9732 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:22.963274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833ddeb-900c-4078-83cc-b87fa63c9732 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:22.970518 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833ddeb-900c-4078-83cc-b87fa63c9732 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:22.987998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1096/2161] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:50:23.209058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:24.482570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cbd11-de64-4c37-922b-1528629ef801 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:24.483712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cbd11-de64-4c37-922b-1528629ef801 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:24.483712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cbd11-de64-4c37-922b-1528629ef801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:24.513428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d06cbd11-de64-4c37-922b-1528629ef801 None None] MFA Rules not processed for user `4d941fd5e5fd4eb281fc92a71c32d6f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:24.514831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d06cbd11-de64-4c37-922b-1528629ef801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:24.609694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1066/2162] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:50:24.625433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833ddeb-900c-4078-83cc-b87fa63c9732 service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:24.637148 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833ddeb-900c-4078-83cc-b87fa63c9732 service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:24.643058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafbe41-2534-422c-9430-a4c393eb265b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:24.644001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafbe41-2534-422c-9430-a4c393eb265b service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:24.644773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafbe41-2534-422c-9430-a4c393eb265b service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:24.645187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafbe41-2534-422c-9430-a4c393eb265b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:24.710456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafbe41-2534-422c-9430-a4c393eb265b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:24.712706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafbe41-2534-422c-9430-a4c393eb265b service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:24.724451 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1097/2163] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:50:24.794879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:26.010777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:26.414893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c65838-ea40-4160-be0c-b996ccf2a80a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:26.416770 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c65838-ea40-4160-be0c-b996ccf2a80a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:26.417081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c65838-ea40-4160-be0c-b996ccf2a80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:26.441508 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d1c65838-ea40-4160-be0c-b996ccf2a80a None None] MFA Rules not processed for user `00491255f4b44125b6dc9f816c9bbb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:26.442849 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d1c65838-ea40-4160-be0c-b996ccf2a80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:26.452771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1067/2164] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:27.482885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:28.312502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafbe41-2534-422c-9430-a4c393eb265b service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:28.396945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafbe41-2534-422c-9430-a4c393eb265b service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:28.402482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565d326-1097-4c81-b8a0-6ad14ea76f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:28.410192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565d326-1097-4c81-b8a0-6ad14ea76f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:28.410192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565d326-1097-4c81-b8a0-6ad14ea76f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:28.410192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565d326-1097-4c81-b8a0-6ad14ea76f47 admin admin] PATH_INFO: `/v3/users/c5d80285241948a8859fcd9e1b516da7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:28.410192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565d326-1097-4c81-b8a0-6ad14ea76f47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5d80285241948a8859fcd9e1b516da7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:28.410192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565d326-1097-4c81-b8a0-6ad14ea76f47 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:28.455721 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:28.678034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a565d326-1097-4c81-b8a0-6ad14ea76f47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5d80285241948a8859fcd9e1b516da7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:28.735728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a565d326-1097-4c81-b8a0-6ad14ea76f47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5d80285241948a8859fcd9e1b516da7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:28.736774 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a565d326-1097-4c81-b8a0-6ad14ea76f47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5d80285241948a8859fcd9e1b516da7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:28.751685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a565d326-1097-4c81-b8a0-6ad14ea76f47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5d80285241948a8859fcd9e1b516da7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:28.775198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a565d326-1097-4c81-b8a0-6ad14ea76f47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5d80285241948a8859fcd9e1b516da7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:28.789952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1098/2165] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:50:28 2022] DELETE /identity/v3/users/c5d80285241948a8859fcd9e1b516da7 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:28.795917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9fbc4-9961-479e-b6f4-9c6148cd8124 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:28.797090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9fbc4-9961-479e-b6f4-9c6148cd8124 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:28.797090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9fbc4-9961-479e-b6f4-9c6148cd8124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:28.829065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-fac9fbc4-9961-479e-b6f4-9c6148cd8124 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:28.830402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fac9fbc4-9961-479e-b6f4-9c6148cd8124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:28.898323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6c7ac-682d-4128-a1e7-c99ec2ac19c6 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:28.899011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6c7ac-682d-4128-a1e7-c99ec2ac19c6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:28.899447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6c7ac-682d-4128-a1e7-c99ec2ac19c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:28.923204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-95e6c7ac-682d-4128-a1e7-c99ec2ac19c6 None None] MFA Rules not processed for user `dc285681bd4843028f0fbbbd01ff4adb`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:28.925460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-95e6c7ac-682d-4128-a1e7-c99ec2ac19c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:28.936470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1068/2166] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:28.949690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9fbc4-9961-479e-b6f4-9c6148cd8124 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:28.953188 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fac9fbc4-9961-479e-b6f4-9c6148cd8124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:29.030155 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1099/2167] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:50:29.067456 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9fbc4-9961-479e-b6f4-9c6148cd8124 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:29.089431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13037fcc-41d9-4625-b6db-8c3c4c29b774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:29.090489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13037fcc-41d9-4625-b6db-8c3c4c29b774 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:29.092913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13037fcc-41d9-4625-b6db-8c3c4c29b774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:29.093465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13037fcc-41d9-4625-b6db-8c3c4c29b774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:29.118853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13037fcc-41d9-4625-b6db-8c3c4c29b774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:29.120571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13037fcc-41d9-4625-b6db-8c3c4c29b774 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:29.149764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1069/2168] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:50:29.465576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:30.181579 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6c7ac-682d-4128-a1e7-c99ec2ac19c6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:30.183072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-95e6c7ac-682d-4128-a1e7-c99ec2ac19c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:30.205713 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6c7ac-682d-4128-a1e7-c99ec2ac19c6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:30.211042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:30.212102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:30.212528 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:30.212942 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] PATH_INFO: `/v3/projects/f5372d813a84489ebb1d95a56a24b609` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:30.214777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5372d813a84489ebb1d95a56a24b609)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:30.217163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:30.345254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5372d813a84489ebb1d95a56a24b609', 'request_id': 'req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:30.351013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5372d813a84489ebb1d95a56a24b609', 'request_id': 'req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:30.359773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5372d813a84489ebb1d95a56a24b609', 'request_id': 'req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:30.424933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] The token cache is being invalidate because project f5372d813a84489ebb1d95a56a24b609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:50:30.425515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:30.427697 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:30.430862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1100/2169] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:50:30 2022] DELETE /identity/v3/projects/f5372d813a84489ebb1d95a56a24b609 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:30.436426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13037fcc-41d9-4625-b6db-8c3c4c29b774 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:30.447047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13037fcc-41d9-4625-b6db-8c3c4c29b774 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:30.530485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d414f-87a9-470c-8da9-b014d91a9401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:30.531419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d414f-87a9-470c-8da9-b014d91a9401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:30.533081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d414f-87a9-470c-8da9-b014d91a9401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:30.533081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d414f-87a9-470c-8da9-b014d91a9401 admin admin] PATH_INFO: `/v3/users/b3f37a8ef4d64bdd80a17bd43ad3f81a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:30.533478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d414f-87a9-470c-8da9-b014d91a9401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f37a8ef4d64bdd80a17bd43ad3f81a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:30.535597 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d414f-87a9-470c-8da9-b014d91a9401 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:30.805917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5b8d414f-87a9-470c-8da9-b014d91a9401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f37a8ef4d64bdd80a17bd43ad3f81a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:30.860971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5b8d414f-87a9-470c-8da9-b014d91a9401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f37a8ef4d64bdd80a17bd43ad3f81a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:30.861701 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5b8d414f-87a9-470c-8da9-b014d91a9401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f37a8ef4d64bdd80a17bd43ad3f81a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:30.874508 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5b8d414f-87a9-470c-8da9-b014d91a9401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f37a8ef4d64bdd80a17bd43ad3f81a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:30.882231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5b8d414f-87a9-470c-8da9-b014d91a9401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f37a8ef4d64bdd80a17bd43ad3f81a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:30.888495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1070/2170] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:50:30 2022] DELETE /identity/v3/users/b3f37a8ef4d64bdd80a17bd43ad3f81a => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:31.387956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:31.427468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:31.428919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:31.444268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0448448-bb52-473f-8dcf-c0b1e2e7b19a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:31.533954 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fbac3-9c57-4cb2-8422-300c8e963313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:31.534473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fbac3-9c57-4cb2-8422-300c8e963313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:31.534749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fbac3-9c57-4cb2-8422-300c8e963313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:31.535021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fbac3-9c57-4cb2-8422-300c8e963313 admin admin] PATH_INFO: `/v3/projects/0fef2b8b86b341ad8904e5668819b8aa` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:31.536197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fbac3-9c57-4cb2-8422-300c8e963313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fef2b8b86b341ad8904e5668819b8aa)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:31.538162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fbac3-9c57-4cb2-8422-300c8e963313 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:31.678455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4a3fbac3-9c57-4cb2-8422-300c8e963313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fef2b8b86b341ad8904e5668819b8aa', 'request_id': 'req-4a3fbac3-9c57-4cb2-8422-300c8e963313'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:31.685097 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4a3fbac3-9c57-4cb2-8422-300c8e963313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fef2b8b86b341ad8904e5668819b8aa', 'request_id': 'req-4a3fbac3-9c57-4cb2-8422-300c8e963313'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:31.692069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4a3fbac3-9c57-4cb2-8422-300c8e963313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fef2b8b86b341ad8904e5668819b8aa', 'request_id': 'req-4a3fbac3-9c57-4cb2-8422-300c8e963313'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:31.800998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4a3fbac3-9c57-4cb2-8422-300c8e963313 admin admin] The token cache is being invalidate because project 0fef2b8b86b341ad8904e5668819b8aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:50:31.801290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4a3fbac3-9c57-4cb2-8422-300c8e963313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:31.803267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-4a3fbac3-9c57-4cb2-8422-300c8e963313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:31.804840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1101/2171] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:50:31 2022] DELETE /identity/v3/projects/0fef2b8b86b341ad8904e5668819b8aa => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:31.810788 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36773c-47c6-4efe-b4ac-3257aba01c4c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:31.811972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36773c-47c6-4efe-b4ac-3257aba01c4c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:31.812815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36773c-47c6-4efe-b4ac-3257aba01c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:31.859804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7c36773c-47c6-4efe-b4ac-3257aba01c4c None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:31.861899 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7c36773c-47c6-4efe-b4ac-3257aba01c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:31.937677 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1071/2172] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:31.942840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee55d0-12e4-4dc5-8373-ba93135b3403 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:31.943175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee55d0-12e4-4dc5-8373-ba93135b3403 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:31.944728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee55d0-12e4-4dc5-8373-ba93135b3403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:31.988711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-fbee55d0-12e4-4dc5-8373-ba93135b3403 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:31.990348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fbee55d0-12e4-4dc5-8373-ba93135b3403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:32.000173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1102/2173] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:32.006260 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36773c-47c6-4efe-b4ac-3257aba01c4c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:32.016470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:32.018729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36773c-47c6-4efe-b4ac-3257aba01c4c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:32.024029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5668e-17e0-486d-b30b-101c41bdcb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:32.024709 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5668e-17e0-486d-b30b-101c41bdcb83 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:32.025394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5668e-17e0-486d-b30b-101c41bdcb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:32.026302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5668e-17e0-486d-b30b-101c41bdcb83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:32.026629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5668e-17e0-486d-b30b-101c41bdcb83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:50:32.027022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5668e-17e0-486d-b30b-101c41bdcb83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:32.028938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5668e-17e0-486d-b30b-101c41bdcb83 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:32.046661 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1072/2174] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:50:32.048408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee55d0-12e4-4dc5-8373-ba93135b3403 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:32.058733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee55d0-12e4-4dc5-8373-ba93135b3403 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:32.082456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8f3f4-3637-45d9-a334-0767fd1d7476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:32.083435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8f3f4-3637-45d9-a334-0767fd1d7476 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:32.084116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8f3f4-3637-45d9-a334-0767fd1d7476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:32.084944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8f3f4-3637-45d9-a334-0767fd1d7476 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:32.085710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8f3f4-3637-45d9-a334-0767fd1d7476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:32.088124 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8f3f4-3637-45d9-a334-0767fd1d7476 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:32.193766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1103/2175] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:50:32 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:50:32.197184 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5668e-17e0-486d-b30b-101c41bdcb83 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:32.208074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5668e-17e0-486d-b30b-101c41bdcb83 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:32.272152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624592e-78af-43a7-a3e8-3fcd70a5eb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:32.273321 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624592e-78af-43a7-a3e8-3fcd70a5eb05 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:32.273321 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624592e-78af-43a7-a3e8-3fcd70a5eb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:32.273321 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624592e-78af-43a7-a3e8-3fcd70a5eb05 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:32.273998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624592e-78af-43a7-a3e8-3fcd70a5eb05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:32.276538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624592e-78af-43a7-a3e8-3fcd70a5eb05 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:32.420181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1073/2176] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:50:32 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:32.421485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8f3f4-3637-45d9-a334-0767fd1d7476 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:32.436637 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8f3f4-3637-45d9-a334-0767fd1d7476 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:32.448961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3bcc9-d63f-458f-a0bb-50dd4ae26a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:32.448961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3bcc9-d63f-458f-a0bb-50dd4ae26a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:32.448961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3bcc9-d63f-458f-a0bb-50dd4ae26a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:32.448961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3bcc9-d63f-458f-a0bb-50dd4ae26a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:32.449278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3bcc9-d63f-458f-a0bb-50dd4ae26a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:32.450750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3bcc9-d63f-458f-a0bb-50dd4ae26a60 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:32.470927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1104/2177] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:32.475565 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624592e-78af-43a7-a3e8-3fcd70a5eb05 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:32.486553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624592e-78af-43a7-a3e8-3fcd70a5eb05 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:32.491300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae4edf-62e9-4003-93d6-b952120a96fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:32.491897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae4edf-62e9-4003-93d6-b952120a96fe admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:32.492177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae4edf-62e9-4003-93d6-b952120a96fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:32.492449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae4edf-62e9-4003-93d6-b952120a96fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:32.493096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae4edf-62e9-4003-93d6-b952120a96fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:32.494715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae4edf-62e9-4003-93d6-b952120a96fe admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:32.506400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1074/2178] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:32.510601 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3bcc9-d63f-458f-a0bb-50dd4ae26a60 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:32.521388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3bcc9-d63f-458f-a0bb-50dd4ae26a60 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:32.528177 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2c46a-e863-414c-8c29-9ba7ff434878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:32.528827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2c46a-e863-414c-8c29-9ba7ff434878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:32.529134 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2c46a-e863-414c-8c29-9ba7ff434878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:32.529407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2c46a-e863-414c-8c29-9ba7ff434878 admin admin] PATH_INFO: `/v3/projects/474bde21ab8144008c7dc94d1933065a/users/53711b6d34214d3e93fd58e0f20e3134/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:32.545055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2c46a-e863-414c-8c29-9ba7ff434878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474bde21ab8144008c7dc94d1933065a, user_id=53711b6d34214d3e93fd58e0f20e3134, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:32.548506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2c46a-e863-414c-8c29-9ba7ff434878 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:32.653931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1105/2179] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:32 2022] PUT /identity/v3/projects/474bde21ab8144008c7dc94d1933065a/users/53711b6d34214d3e93fd58e0f20e3134/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:32.665364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9ecd-41a8-4a54-94a3-6d48ae6f0b20 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:32.665677 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9ecd-41a8-4a54-94a3-6d48ae6f0b20 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:32.665980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9ecd-41a8-4a54-94a3-6d48ae6f0b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:32.695866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-269e9ecd-41a8-4a54-94a3-6d48ae6f0b20 None None] MFA Rules not processed for user `53711b6d34214d3e93fd58e0f20e3134`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:32.697092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-269e9ecd-41a8-4a54-94a3-6d48ae6f0b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:32.762936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:32.768499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1075/2180] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:32.780367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2c46a-e863-414c-8c29-9ba7ff434878 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:32.782129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-85e2c46a-e863-414c-8c29-9ba7ff434878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:32.843937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2c46a-e863-414c-8c29-9ba7ff434878 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:32.847692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa75e32-c920-4050-a27c-295121576d94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:32.848296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa75e32-c920-4050-a27c-295121576d94 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:32.848577 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa75e32-c920-4050-a27c-295121576d94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:32.848851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa75e32-c920-4050-a27c-295121576d94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:32.857826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa75e32-c920-4050-a27c-295121576d94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:32.859815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa75e32-c920-4050-a27c-295121576d94 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:32.869402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1106/2181] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:50:32.889042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9ecd-41a8-4a54-94a3-6d48ae6f0b20 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:32.970619 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9ecd-41a8-4a54-94a3-6d48ae6f0b20 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:32.976032 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:32.976665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:32.976948 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:32.977225 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] PATH_INFO: `/v3/users/53711b6d34214d3e93fd58e0f20e3134` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:32.978548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] RBAC: Authorizing `identity:delete_user(user_id=53711b6d34214d3e93fd58e0f20e3134)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:32.980518 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:33.180360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53711b6d34214d3e93fd58e0f20e3134'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:33.305336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53711b6d34214d3e93fd58e0f20e3134'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:33.306029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53711b6d34214d3e93fd58e0f20e3134'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:33.313957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53711b6d34214d3e93fd58e0f20e3134'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:33.325831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53711b6d34214d3e93fd58e0f20e3134'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:33.332161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1076/2182] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:50:32 2022] DELETE /identity/v3/users/53711b6d34214d3e93fd58e0f20e3134 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:33.336758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88452199-fa17-4dd3-8af3-7001bbfb213b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:33.337048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88452199-fa17-4dd3-8af3-7001bbfb213b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:33.337326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88452199-fa17-4dd3-8af3-7001bbfb213b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:33.356652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-88452199-fa17-4dd3-8af3-7001bbfb213b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:33.357933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-88452199-fa17-4dd3-8af3-7001bbfb213b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:33.416003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1107/2183] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:33.431175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:33.435048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:33.502676 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1321566-9aaf-4038-9102-32c02390369a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:33.506595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3afab-e046-462b-b8e5-1020b0b836b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:33.507304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3afab-e046-462b-b8e5-1020b0b836b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:33.507657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3afab-e046-462b-b8e5-1020b0b836b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:33.508025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3afab-e046-462b-b8e5-1020b0b836b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:33.520204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3afab-e046-462b-b8e5-1020b0b836b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:33.521935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3afab-e046-462b-b8e5-1020b0b836b9 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:33.532936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1077/2184] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:50:33.672449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:33.953364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88452199-fa17-4dd3-8af3-7001bbfb213b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:33.955238 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-88452199-fa17-4dd3-8af3-7001bbfb213b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:33.975345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88452199-fa17-4dd3-8af3-7001bbfb213b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:33.984315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d059a5-8c06-4cf8-a26e-98d06671c3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:33.985320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d059a5-8c06-4cf8-a26e-98d06671c3af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:33.986099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d059a5-8c06-4cf8-a26e-98d06671c3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:33.986598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d059a5-8c06-4cf8-a26e-98d06671c3af admin admin] PATH_INFO: `/v3/projects/474bde21ab8144008c7dc94d1933065a` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:33.988737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d059a5-8c06-4cf8-a26e-98d06671c3af admin admin] RBAC: Authorizing `identity:delete_project(project_id=474bde21ab8144008c7dc94d1933065a)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:33.992421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d059a5-8c06-4cf8-a26e-98d06671c3af admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:34.198953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-31d059a5-8c06-4cf8-a26e-98d06671c3af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474bde21ab8144008c7dc94d1933065a', 'request_id': 'req-31d059a5-8c06-4cf8-a26e-98d06671c3af'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:34.204425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-31d059a5-8c06-4cf8-a26e-98d06671c3af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474bde21ab8144008c7dc94d1933065a', 'request_id': 'req-31d059a5-8c06-4cf8-a26e-98d06671c3af'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:34.211404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-31d059a5-8c06-4cf8-a26e-98d06671c3af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474bde21ab8144008c7dc94d1933065a', 'request_id': 'req-31d059a5-8c06-4cf8-a26e-98d06671c3af'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:34.326963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-31d059a5-8c06-4cf8-a26e-98d06671c3af admin admin] The token cache is being invalidate because project 474bde21ab8144008c7dc94d1933065a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:50:34.327460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-31d059a5-8c06-4cf8-a26e-98d06671c3af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:34.328672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-31d059a5-8c06-4cf8-a26e-98d06671c3af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:34.331199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1108/2185] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:50:33 2022] DELETE /identity/v3/projects/474bde21ab8144008c7dc94d1933065a => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:50:34.337893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b89f12-70de-4bac-a7c1-7254433b4b69 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:34.338395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b89f12-70de-4bac-a7c1-7254433b4b69 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:34.338505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b89f12-70de-4bac-a7c1-7254433b4b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:34.359673 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-83b89f12-70de-4bac-a7c1-7254433b4b69 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:34.360999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-83b89f12-70de-4bac-a7c1-7254433b4b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:34.428084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1078/2186] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:34.438939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df65fb9-913b-49d4-a431-06a7af89f687 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:34.442154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df65fb9-913b-49d4-a431-06a7af89f687 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:34.442154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df65fb9-913b-49d4-a431-06a7af89f687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:34.473208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0df65fb9-913b-49d4-a431-06a7af89f687 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:34.474447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0df65fb9-913b-49d4-a431-06a7af89f687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:34.486332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1109/2187] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:34.487576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b89f12-70de-4bac-a7c1-7254433b4b69 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:34.497655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b89f12-70de-4bac-a7c1-7254433b4b69 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:34.503222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a15f57-c579-4414-9bcf-b05dbbd7b64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:34.503929 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a15f57-c579-4414-9bcf-b05dbbd7b64e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:34.504329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a15f57-c579-4414-9bcf-b05dbbd7b64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:34.504602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a15f57-c579-4414-9bcf-b05dbbd7b64e admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:34.505348 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a15f57-c579-4414-9bcf-b05dbbd7b64e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:50:34.505773 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a15f57-c579-4414-9bcf-b05dbbd7b64e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:34.507439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a15f57-c579-4414-9bcf-b05dbbd7b64e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:34.522671 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1079/2188] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:50:34.527389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df65fb9-913b-49d4-a431-06a7af89f687 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:34.540089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df65fb9-913b-49d4-a431-06a7af89f687 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:34.546228 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7ad01-62c8-48df-984a-7873bd733d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:34.546864 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7ad01-62c8-48df-984a-7873bd733d9e admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:34.547150 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7ad01-62c8-48df-984a-7873bd733d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:34.547425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7ad01-62c8-48df-984a-7873bd733d9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:34.548069 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7ad01-62c8-48df-984a-7873bd733d9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:34.550373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7ad01-62c8-48df-984a-7873bd733d9e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:34.662064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1110/2189] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:50:34 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:50:34.663472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a15f57-c579-4414-9bcf-b05dbbd7b64e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:34.673643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a15f57-c579-4414-9bcf-b05dbbd7b64e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:34.729272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:34.734784 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393c94f-28b1-4305-9644-fb3632c20c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:34.735448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393c94f-28b1-4305-9644-fb3632c20c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:34.735728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393c94f-28b1-4305-9644-fb3632c20c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:34.736853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393c94f-28b1-4305-9644-fb3632c20c72 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:34.736853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393c94f-28b1-4305-9644-fb3632c20c72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:34.738635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393c94f-28b1-4305-9644-fb3632c20c72 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:34.830869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1080/2190] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:50:34 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:34.835361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7ad01-62c8-48df-984a-7873bd733d9e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:34.846304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7ad01-62c8-48df-984a-7873bd733d9e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:34.851124 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a50b9-02dd-4ff4-b5e4-6d1b7d94e6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:34.852162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a50b9-02dd-4ff4-b5e4-6d1b7d94e6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:34.852162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a50b9-02dd-4ff4-b5e4-6d1b7d94e6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:34.854201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a50b9-02dd-4ff4-b5e4-6d1b7d94e6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:34.854201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a50b9-02dd-4ff4-b5e4-6d1b7d94e6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:34.858985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a50b9-02dd-4ff4-b5e4-6d1b7d94e6c1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:34.870557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1111/2191] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:34.874938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393c94f-28b1-4305-9644-fb3632c20c72 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:34.886213 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393c94f-28b1-4305-9644-fb3632c20c72 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:34.892464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a4fb4-d3b4-4702-a919-51807d8886a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:34.893078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4fb4-d3b4-4702-a919-51807d8886a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:34.893355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4fb4-d3b4-4702-a919-51807d8886a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:34.900782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4fb4-d3b4-4702-a919-51807d8886a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:34.900782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a4fb4-d3b4-4702-a919-51807d8886a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:34.900782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a4fb4-d3b4-4702-a919-51807d8886a9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:34.907738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1081/2192] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:34.912144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a50b9-02dd-4ff4-b5e4-6d1b7d94e6c1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:34.923949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a50b9-02dd-4ff4-b5e4-6d1b7d94e6c1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:34.929752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a73ef-63f1-4d42-9a03-2268cfb3e993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:34.939868 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a73ef-63f1-4d42-9a03-2268cfb3e993 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:34.939868 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a73ef-63f1-4d42-9a03-2268cfb3e993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:34.939868 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a73ef-63f1-4d42-9a03-2268cfb3e993 admin admin] PATH_INFO: `/v3/projects/fb9bb6c2a11f4840875d2882cd5a3df2/users/6fa72d74b6b24ee997ec04eb7f69feee/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:34.947382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a73ef-63f1-4d42-9a03-2268cfb3e993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb9bb6c2a11f4840875d2882cd5a3df2, user_id=6fa72d74b6b24ee997ec04eb7f69feee, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:34.950163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a73ef-63f1-4d42-9a03-2268cfb3e993 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:35.041252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1112/2193] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:34 2022] PUT /identity/v3/projects/fb9bb6c2a11f4840875d2882cd5a3df2/users/6fa72d74b6b24ee997ec04eb7f69feee/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:35.054571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30215d5-2a56-47f4-b554-3e9eada86144 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:35.055064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30215d5-2a56-47f4-b554-3e9eada86144 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:35.055511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30215d5-2a56-47f4-b554-3e9eada86144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:35.096754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a30215d5-2a56-47f4-b554-3e9eada86144 None None] MFA Rules not processed for user `6fa72d74b6b24ee997ec04eb7f69feee`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:35.098049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a30215d5-2a56-47f4-b554-3e9eada86144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:35.218028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1082/2194] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:35.472757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:36.253789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:36.300430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:36.820849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:37.021983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:38.219793 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:38.397712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:39.655198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:39.906407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:40.498694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a73ef-63f1-4d42-9a03-2268cfb3e993 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:40.505289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-363a73ef-63f1-4d42-9a03-2268cfb3e993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:40.565712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:40.569372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a73ef-63f1-4d42-9a03-2268cfb3e993 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:40.574289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:40.574917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:40.575203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:40.575476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] PATH_INFO: `/v3/users/7bf23c2b720b44b5954b80941443a49c` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:40.576627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf23c2b720b44b5954b80941443a49c)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:40.578658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:40.777603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf23c2b720b44b5954b80941443a49c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:40.849090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf23c2b720b44b5954b80941443a49c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:40.849730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf23c2b720b44b5954b80941443a49c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:40.857413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf23c2b720b44b5954b80941443a49c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:40.870474 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf23c2b720b44b5954b80941443a49c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:40.876340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1113/2195] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:50:40 2022] DELETE /identity/v3/users/7bf23c2b720b44b5954b80941443a49c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:40.880815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79809846-119a-4add-8a24-bcfc033325af None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:40.881185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79809846-119a-4add-8a24-bcfc033325af None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:40.881461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79809846-119a-4add-8a24-bcfc033325af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:40.902082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-79809846-119a-4add-8a24-bcfc033325af None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:40.904920 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-79809846-119a-4add-8a24-bcfc033325af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:40.977823 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1083/2196] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:40.993670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:40.995531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:41.030273 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:41.057544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f22d45-ca90-4cde-b5dd-ee16e854b41f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:41.061917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64403f2c-d186-49a8-bca1-b7a9613a3c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:41.062521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64403f2c-d186-49a8-bca1-b7a9613a3c85 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:41.062802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64403f2c-d186-49a8-bca1-b7a9613a3c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:41.063074 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64403f2c-d186-49a8-bca1-b7a9613a3c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:41.076806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64403f2c-d186-49a8-bca1-b7a9613a3c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:41.078687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64403f2c-d186-49a8-bca1-b7a9613a3c85 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:41.096777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1114/2197] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:50:41.113885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:41.303098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:41.419002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:41.558959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79809846-119a-4add-8a24-bcfc033325af None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:41.561289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-79809846-119a-4add-8a24-bcfc033325af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:41.591631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79809846-119a-4add-8a24-bcfc033325af None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:41.598008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005d14e-7f61-4615-9b7c-06600425c47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:41.599313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005d14e-7f61-4615-9b7c-06600425c47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:41.599485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005d14e-7f61-4615-9b7c-06600425c47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:41.599718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005d14e-7f61-4615-9b7c-06600425c47f admin admin] PATH_INFO: `/v3/projects/75f38533dcd549aa8226b539e83715b4` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:41.601233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005d14e-7f61-4615-9b7c-06600425c47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f38533dcd549aa8226b539e83715b4)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:41.603254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005d14e-7f61-4615-9b7c-06600425c47f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:41.636892 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:41.710798 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:41.757357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1005d14e-7f61-4615-9b7c-06600425c47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f38533dcd549aa8226b539e83715b4', 'request_id': 'req-1005d14e-7f61-4615-9b7c-06600425c47f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:41.764208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1005d14e-7f61-4615-9b7c-06600425c47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f38533dcd549aa8226b539e83715b4', 'request_id': 'req-1005d14e-7f61-4615-9b7c-06600425c47f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:41.779027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1005d14e-7f61-4615-9b7c-06600425c47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f38533dcd549aa8226b539e83715b4', 'request_id': 'req-1005d14e-7f61-4615-9b7c-06600425c47f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:41.865892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1005d14e-7f61-4615-9b7c-06600425c47f admin admin] The token cache is being invalidate because project 75f38533dcd549aa8226b539e83715b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:50:41.866186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1005d14e-7f61-4615-9b7c-06600425c47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:41.866809 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1005d14e-7f61-4615-9b7c-06600425c47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:41.868537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1084/2198] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:50:41 2022] DELETE /identity/v3/projects/75f38533dcd549aa8226b539e83715b4 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:41.874973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb65f7-2878-4b01-8003-b368f80b81ad None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:41.874973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb65f7-2878-4b01-8003-b368f80b81ad None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:41.874973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb65f7-2878-4b01-8003-b368f80b81ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:41.897271 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d6bb65f7-2878-4b01-8003-b368f80b81ad None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:41.898553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d6bb65f7-2878-4b01-8003-b368f80b81ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:41.966253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1115/2199] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:50:41.968456 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:41.974708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60b95f-af91-4f22-8370-b67318f6ab75 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:41.981457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60b95f-af91-4f22-8370-b67318f6ab75 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:41.981457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60b95f-af91-4f22-8370-b67318f6ab75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:41.996916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3d60b95f-af91-4f22-8370-b67318f6ab75 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:41.998207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3d60b95f-af91-4f22-8370-b67318f6ab75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:42.002343 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:42.007113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1085/2200] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:42.011461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb65f7-2878-4b01-8003-b368f80b81ad None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:42.022307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb65f7-2878-4b01-8003-b368f80b81ad None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:42.027206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d577de-c7ac-4a34-bf57-0d6cab31156c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:42.027817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d577de-c7ac-4a34-bf57-0d6cab31156c admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:42.028114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d577de-c7ac-4a34-bf57-0d6cab31156c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:42.028368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d577de-c7ac-4a34-bf57-0d6cab31156c admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:42.029675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d577de-c7ac-4a34-bf57-0d6cab31156c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:50:42.030020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d577de-c7ac-4a34-bf57-0d6cab31156c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:42.031818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d577de-c7ac-4a34-bf57-0d6cab31156c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:42.044599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1116/2201] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:50:42.049766 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60b95f-af91-4f22-8370-b67318f6ab75 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:42.059128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60b95f-af91-4f22-8370-b67318f6ab75 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:42.063854 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f554ec-3665-4fad-ab1f-0d9f1363f584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:42.064450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f554ec-3665-4fad-ab1f-0d9f1363f584 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:42.068183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f554ec-3665-4fad-ab1f-0d9f1363f584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:42.068183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f554ec-3665-4fad-ab1f-0d9f1363f584 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:42.068183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f554ec-3665-4fad-ab1f-0d9f1363f584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:42.068183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f554ec-3665-4fad-ab1f-0d9f1363f584 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:42.087512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:42.151062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1086/2202] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:50:42 2022] POST /identity/v3/projects => generated 399 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:42.156454 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d577de-c7ac-4a34-bf57-0d6cab31156c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:42.170640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d577de-c7ac-4a34-bf57-0d6cab31156c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:42.208364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:42.230931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a515b2-15bf-48a3-b9e2-60da8e7c3827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:42.231549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a515b2-15bf-48a3-b9e2-60da8e7c3827 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:42.231901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a515b2-15bf-48a3-b9e2-60da8e7c3827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:42.232131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a515b2-15bf-48a3-b9e2-60da8e7c3827 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:42.233017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a515b2-15bf-48a3-b9e2-60da8e7c3827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:42.238122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a515b2-15bf-48a3-b9e2-60da8e7c3827 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:42.242059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:42.352054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1117/2203] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:50:42 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:50:42.357182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f554ec-3665-4fad-ab1f-0d9f1363f584 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:42.368171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f554ec-3665-4fad-ab1f-0d9f1363f584 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:42.373224 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4774d-34f8-4699-aaa2-8a130b487fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:42.373930 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4774d-34f8-4699-aaa2-8a130b487fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:42.374234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4774d-34f8-4699-aaa2-8a130b487fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:42.374640 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4774d-34f8-4699-aaa2-8a130b487fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:42.375296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4774d-34f8-4699-aaa2-8a130b487fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:42.376893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4774d-34f8-4699-aaa2-8a130b487fdc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:42.387745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1087/2204] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:50:42.392780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a515b2-15bf-48a3-b9e2-60da8e7c3827 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:42.403292 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a515b2-15bf-48a3-b9e2-60da8e7c3827 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:42.408937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59463d40-1cc1-49de-9c68-a8afb115dcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:42.409619 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463d40-1cc1-49de-9c68-a8afb115dcbb admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:42.409972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463d40-1cc1-49de-9c68-a8afb115dcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:42.410280 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463d40-1cc1-49de-9c68-a8afb115dcbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:42.412284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59463d40-1cc1-49de-9c68-a8afb115dcbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:42.413109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59463d40-1cc1-49de-9c68-a8afb115dcbb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:42.431914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1118/2205] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:50:42.434611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4774d-34f8-4699-aaa2-8a130b487fdc admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:42.445734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4774d-34f8-4699-aaa2-8a130b487fdc admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:42.452394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239f26c-8c73-48fa-a592-d339173c67d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:42.452923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239f26c-8c73-48fa-a592-d339173c67d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:42.453206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239f26c-8c73-48fa-a592-d339173c67d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:42.453478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239f26c-8c73-48fa-a592-d339173c67d6 admin admin] PATH_INFO: `/v3/projects/be6c72b2390a4bff9e5fdad9e768613a/users/8ed989c11151402da129fb472d131ad7/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:42.469696 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239f26c-8c73-48fa-a592-d339173c67d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be6c72b2390a4bff9e5fdad9e768613a, user_id=8ed989c11151402da129fb472d131ad7, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:42.472397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239f26c-8c73-48fa-a592-d339173c67d6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:42.550980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1088/2206] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:50:42 2022] PUT /identity/v3/projects/be6c72b2390a4bff9e5fdad9e768613a/users/8ed989c11151402da129fb472d131ad7/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:42.562289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227431c-4e85-4b2b-93dc-ebad8a249423 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:42.562578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227431c-4e85-4b2b-93dc-ebad8a249423 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:42.562853 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227431c-4e85-4b2b-93dc-ebad8a249423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:42.593654 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b227431c-4e85-4b2b-93dc-ebad8a249423 None None] MFA Rules not processed for user `8ed989c11151402da129fb472d131ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:42.594881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b227431c-4e85-4b2b-93dc-ebad8a249423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:42.625690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:42.661951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1119/2207] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:42 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:50:42.675674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239f26c-8c73-48fa-a592-d339173c67d6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:42.677838 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a239f26c-8c73-48fa-a592-d339173c67d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:42.683858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:42.765898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239f26c-8c73-48fa-a592-d339173c67d6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:42.768667 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:42.771830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ca2d2-13d5-494f-8e97-afe94de582b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:42.772442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ca2d2-13d5-494f-8e97-afe94de582b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:42.772850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ca2d2-13d5-494f-8e97-afe94de582b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:42.772992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ca2d2-13d5-494f-8e97-afe94de582b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:42.782324 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ca2d2-13d5-494f-8e97-afe94de582b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:42.783872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ca2d2-13d5-494f-8e97-afe94de582b7 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:42.793051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1089/2208] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:42 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:50:42.884403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:52.091998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227431c-4e85-4b2b-93dc-ebad8a249423 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:52.177895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227431c-4e85-4b2b-93dc-ebad8a249423 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:52.183045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:52.183610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:52.183893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:52.184183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] PATH_INFO: `/v3/users/8ed989c11151402da129fb472d131ad7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:52.185366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ed989c11151402da129fb472d131ad7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:52.187940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:52.379616 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ed989c11151402da129fb472d131ad7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:52.449894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ed989c11151402da129fb472d131ad7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:52.450809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ed989c11151402da129fb472d131ad7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:52.458885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ed989c11151402da129fb472d131ad7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:52.473743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ed989c11151402da129fb472d131ad7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:52.482683 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1120/2209] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:50:52 2022] DELETE /identity/v3/users/8ed989c11151402da129fb472d131ad7 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:50:52.487655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eff3cb-bc78-4d21-bcd7-17249aba2ca8 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:52.488136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eff3cb-bc78-4d21-bcd7-17249aba2ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:52.488274 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eff3cb-bc78-4d21-bcd7-17249aba2ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:52.508441 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-02eff3cb-bc78-4d21-bcd7-17249aba2ca8 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:50:52.509689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-02eff3cb-bc78-4d21-bcd7-17249aba2ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:52.555674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:52.578955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1090/2210] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:50:52.596041 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:52.597498 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:52.627475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:50:52.669796 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3b9aa-e195-47c6-82eb-08e3c6c88b4d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:52.674596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78026fd4-a526-4977-919f-d329eecf8636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:52.675285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78026fd4-a526-4977-919f-d329eecf8636 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:52.679010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78026fd4-a526-4977-919f-d329eecf8636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:52.679010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78026fd4-a526-4977-919f-d329eecf8636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:52.690282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78026fd4-a526-4977-919f-d329eecf8636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:52.692018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78026fd4-a526-4977-919f-d329eecf8636 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:52.703149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1121/2211] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:50:53.000277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eff3cb-bc78-4d21-bcd7-17249aba2ca8 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:53.001674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-02eff3cb-bc78-4d21-bcd7-17249aba2ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:53.066341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eff3cb-bc78-4d21-bcd7-17249aba2ca8 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:53.070780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d20335-cf7d-405c-bb35-a46fda09d46d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a58f9101f2d4785b96f0cc55be4330b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:53.071390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d20335-cf7d-405c-bb35-a46fda09d46d service glance] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:53.078848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d20335-cf7d-405c-bb35-a46fda09d46d service glance] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:53.078848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d20335-cf7d-405c-bb35-a46fda09d46d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:53.078848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d0d20335-cf7d-405c-bb35-a46fda09d46d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:53.136960 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d20335-cf7d-405c-bb35-a46fda09d46d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:53.138429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d20335-cf7d-405c-bb35-a46fda09d46d service glance] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:53.147214 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1091/2212] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:50:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:50:53.183052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78026fd4-a526-4977-919f-d329eecf8636 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:50:53.185164 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-78026fd4-a526-4977-919f-d329eecf8636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:50:53.208620 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78026fd4-a526-4977-919f-d329eecf8636 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:50:53.216140 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfeeef8-74c9-4226-8689-189856d02c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:50:53.218601 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfeeef8-74c9-4226-8689-189856d02c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:50:53.219326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfeeef8-74c9-4226-8689-189856d02c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:50:53.219890 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfeeef8-74c9-4226-8689-189856d02c10 admin admin] PATH_INFO: `/v3/projects/be6c72b2390a4bff9e5fdad9e768613a` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:50:53.221372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfeeef8-74c9-4226-8689-189856d02c10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be6c72b2390a4bff9e5fdad9e768613a)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:50:53.223913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfeeef8-74c9-4226-8689-189856d02c10 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:50:53.377668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ebfeeef8-74c9-4226-8689-189856d02c10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be6c72b2390a4bff9e5fdad9e768613a', 'request_id': 'req-ebfeeef8-74c9-4226-8689-189856d02c10'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:53.383948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ebfeeef8-74c9-4226-8689-189856d02c10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be6c72b2390a4bff9e5fdad9e768613a', 'request_id': 'req-ebfeeef8-74c9-4226-8689-189856d02c10'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:53.403855 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ebfeeef8-74c9-4226-8689-189856d02c10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be6c72b2390a4bff9e5fdad9e768613a', 'request_id': 'req-ebfeeef8-74c9-4226-8689-189856d02c10'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:53.461843 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ebfeeef8-74c9-4226-8689-189856d02c10 admin admin] The token cache is being invalidate because project be6c72b2390a4bff9e5fdad9e768613a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:50:53.462089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ebfeeef8-74c9-4226-8689-189856d02c10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:53.462666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ebfeeef8-74c9-4226-8689-189856d02c10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:50:53.464428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1122/2213] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:50:53 2022] DELETE /identity/v3/projects/be6c72b2390a4bff9e5fdad9e768613a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:51:03.143182 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=56 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:03.457603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:08.070986 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d20335-cf7d-405c-bb35-a46fda09d46d service glance] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:08.076302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d0d20335-cf7d-405c-bb35-a46fda09d46d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:08.086328 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d20335-cf7d-405c-bb35-a46fda09d46d service glance] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:08.143450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:08.144074 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:08.144351 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:08.144682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] PATH_INFO: `/v3/users/b23db9350127496fa138dad824635714` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:08.146513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23db9350127496fa138dad824635714)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:08.148461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:08.436153 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23db9350127496fa138dad824635714'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:08.510771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23db9350127496fa138dad824635714'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:08.511418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23db9350127496fa138dad824635714'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:08.519738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23db9350127496fa138dad824635714'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:08.535104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23db9350127496fa138dad824635714'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:08.546813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1092/2214] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:51:08 2022] DELETE /identity/v3/users/b23db9350127496fa138dad824635714 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:51:08.549247 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402821e-b9e1-47f0-a617-843a61e93b58 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:08.549763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402821e-b9e1-47f0-a617-843a61e93b58 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:08.549878 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402821e-b9e1-47f0-a617-843a61e93b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:08.594868 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a402821e-b9e1-47f0-a617-843a61e93b58 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:08.596451 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a402821e-b9e1-47f0-a617-843a61e93b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:08.679484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1123/2215] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:08.690231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:08.691880 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:08.767906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d3870-21d2-4585-8940-51c2f291582a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:08.771767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857e7c2-73a9-4e01-95c8-084efbebaf3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:08.772509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857e7c2-73a9-4e01-95c8-084efbebaf3f service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:08.772789 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857e7c2-73a9-4e01-95c8-084efbebaf3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:08.773059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857e7c2-73a9-4e01-95c8-084efbebaf3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:08.782974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857e7c2-73a9-4e01-95c8-084efbebaf3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:08.784415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857e7c2-73a9-4e01-95c8-084efbebaf3f service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:08.794824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1093/2216] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:51:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:51:09.277213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402821e-b9e1-47f0-a617-843a61e93b58 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:09.280806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a402821e-b9e1-47f0-a617-843a61e93b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:09.293122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402821e-b9e1-47f0-a617-843a61e93b58 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:09.298765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:09.299661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:09.299937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:09.300212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] PATH_INFO: `/v3/projects/f6ca0e0cd370456d877f228ec50c4dc5` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:09.301431 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ca0e0cd370456d877f228ec50c4dc5)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:09.303630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:09.432470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ca0e0cd370456d877f228ec50c4dc5', 'request_id': 'req-f0962337-8d90-4996-8d55-4862e4b00758'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:09.438584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ca0e0cd370456d877f228ec50c4dc5', 'request_id': 'req-f0962337-8d90-4996-8d55-4862e4b00758'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:09.444776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ca0e0cd370456d877f228ec50c4dc5', 'request_id': 'req-f0962337-8d90-4996-8d55-4862e4b00758'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:09.522209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] The token cache is being invalidate because project f6ca0e0cd370456d877f228ec50c4dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:51:09.522614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:09.523397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:09.525463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1124/2217] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:51:09 2022] DELETE /identity/v3/projects/f6ca0e0cd370456d877f228ec50c4dc5 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:51:15.349183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a080c-83d6-4c37-8f61-4f4001dbd759 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:15.349774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a080c-83d6-4c37-8f61-4f4001dbd759 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:15.349873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:15.350239 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a080c-83d6-4c37-8f61-4f4001dbd759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:15.351449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:15.373014 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7e6a080c-83d6-4c37-8f61-4f4001dbd759 None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:15.374257 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7e6a080c-83d6-4c37-8f61-4f4001dbd759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:15.394650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:15.433811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0962337-8d90-4996-8d55-4862e4b00758 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:15.457779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:15.458971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:15.459682 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:15.460021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] PATH_INFO: `/v3/users/6fa72d74b6b24ee997ec04eb7f69feee` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:15.462745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fa72d74b6b24ee997ec04eb7f69feee)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:15.463908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:15.575110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1094/2218] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:51:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:15.586481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:15.619724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a080c-83d6-4c37-8f61-4f4001dbd759 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:15.621109 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7e6a080c-83d6-4c37-8f61-4f4001dbd759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:15.712958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a080c-83d6-4c37-8f61-4f4001dbd759 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:15.722154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fa501-cc63-464e-b0c4-7ea98c4dfb92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:15.722154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fa501-cc63-464e-b0c4-7ea98c4dfb92 service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:15.722461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fa501-cc63-464e-b0c4-7ea98c4dfb92 service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:15.725258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa72d74b6b24ee997ec04eb7f69feee'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:15.729954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fa501-cc63-464e-b0c4-7ea98c4dfb92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:15.741862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fa501-cc63-464e-b0c4-7ea98c4dfb92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:15.743577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fa501-cc63-464e-b0c4-7ea98c4dfb92 service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:15.754935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1095/2219] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:51:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 18:51:15.914530 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa72d74b6b24ee997ec04eb7f69feee'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:15.915324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa72d74b6b24ee997ec04eb7f69feee'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:15.928120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa72d74b6b24ee997ec04eb7f69feee'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:15.947495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa72d74b6b24ee997ec04eb7f69feee'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:15.958741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1125/2220] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:51:15 2022] DELETE /identity/v3/users/6fa72d74b6b24ee997ec04eb7f69feee => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:51:15.963253 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c134c32-26fd-4f39-901f-3824e6460a26 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:15.963534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c134c32-26fd-4f39-901f-3824e6460a26 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:15.963803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c134c32-26fd-4f39-901f-3824e6460a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:15.998609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-2c134c32-26fd-4f39-901f-3824e6460a26 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:15.999159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2c134c32-26fd-4f39-901f-3824e6460a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:16.088418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1096/2221] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:16.098769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:16.100747 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:16.186314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349aadfb-a5c9-4873-a454-360e4a891ec9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:16.194513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb60a6-fade-4c8f-b4b2-6409916eb20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:16.195373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb60a6-fade-4c8f-b4b2-6409916eb20d service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:16.195732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb60a6-fade-4c8f-b4b2-6409916eb20d service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:16.196225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb60a6-fade-4c8f-b4b2-6409916eb20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:16.219113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb60a6-fade-4c8f-b4b2-6409916eb20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:16.220672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb60a6-fade-4c8f-b4b2-6409916eb20d service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:16.236484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1126/2222] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:51:16.965574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c134c32-26fd-4f39-901f-3824e6460a26 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:16.967722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2c134c32-26fd-4f39-901f-3824e6460a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:16.993477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c134c32-26fd-4f39-901f-3824e6460a26 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:17.002423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:17.003116 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:17.003346 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:17.003633 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0 admin admin] PATH_INFO: `/v3/projects/fb9bb6c2a11f4840875d2882cd5a3df2` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:17.021161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb9bb6c2a11f4840875d2882cd5a3df2)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:17.023119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:17.154481 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb9bb6c2a11f4840875d2882cd5a3df2', 'request_id': 'req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:17.158871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:17.163964 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb9bb6c2a11f4840875d2882cd5a3df2', 'request_id': 'req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:17.176288 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb9bb6c2a11f4840875d2882cd5a3df2', 'request_id': 'req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:17.262297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0 admin admin] The token cache is being invalidate because project fb9bb6c2a11f4840875d2882cd5a3df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:51:17.262642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:17.264997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-5a8aa6eb-e0a7-4c8a-a3af-46ea348b53f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:17.270756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1097/2223] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:51:16 2022] DELETE /identity/v3/projects/fb9bb6c2a11f4840875d2882cd5a3df2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:51:17.272574 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3b2c5-bf87-4d50-8adb-a39b524750f2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:17.273629 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3b2c5-bf87-4d50-8adb-a39b524750f2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:17.274267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3b2c5-bf87-4d50-8adb-a39b524750f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:17.299541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8ae3b2c5-bf87-4d50-8adb-a39b524750f2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:17.300924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8ae3b2c5-bf87-4d50-8adb-a39b524750f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:17.383583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1127/2224] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:17.396489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482a509-34f6-4074-86b0-37bd54b395bf None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:17.396770 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482a509-34f6-4074-86b0-37bd54b395bf None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:17.397041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482a509-34f6-4074-86b0-37bd54b395bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:17.423037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8482a509-34f6-4074-86b0-37bd54b395bf None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:17.424667 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8482a509-34f6-4074-86b0-37bd54b395bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:17.434018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1098/2225] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:17.438952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3b2c5-bf87-4d50-8adb-a39b524750f2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:17.450521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3b2c5-bf87-4d50-8adb-a39b524750f2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:17.456424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e421bf-c6b8-499c-b941-83cbd93a2e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:17.457044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e421bf-c6b8-499c-b941-83cbd93a2e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:17.457345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e421bf-c6b8-499c-b941-83cbd93a2e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:17.457641 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e421bf-c6b8-499c-b941-83cbd93a2e6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:17.458297 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e421bf-c6b8-499c-b941-83cbd93a2e6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:51:17.458597 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e421bf-c6b8-499c-b941-83cbd93a2e6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:17.462504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e421bf-c6b8-499c-b941-83cbd93a2e6a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:17.476935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1128/2226] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:51:17.487935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482a509-34f6-4074-86b0-37bd54b395bf None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:17.498690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482a509-34f6-4074-86b0-37bd54b395bf None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:17.504988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abb119-d8f4-4376-9f93-0a0b3b595505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:17.505729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abb119-d8f4-4376-9f93-0a0b3b595505 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:17.506075 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abb119-d8f4-4376-9f93-0a0b3b595505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:17.506419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abb119-d8f4-4376-9f93-0a0b3b595505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:17.507303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abb119-d8f4-4376-9f93-0a0b3b595505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:17.509276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abb119-d8f4-4376-9f93-0a0b3b595505 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:17.617317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1099/2227] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:51:17 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:51:17.621840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e421bf-c6b8-499c-b941-83cbd93a2e6a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:17.633290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e421bf-c6b8-499c-b941-83cbd93a2e6a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:17.679466 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:17.724513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fab6b-f79d-4595-a64f-50c356df83de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:17.725400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fab6b-f79d-4595-a64f-50c356df83de admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:17.725857 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fab6b-f79d-4595-a64f-50c356df83de admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:17.726268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fab6b-f79d-4595-a64f-50c356df83de admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:17.727273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fab6b-f79d-4595-a64f-50c356df83de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:17.730273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fab6b-f79d-4595-a64f-50c356df83de admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:18.445019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1129/2228] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:51:17 2022] POST /identity/v3/users => generated 293 bytes in 823 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:51:18.449093 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abb119-d8f4-4376-9f93-0a0b3b595505 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:18.460523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abb119-d8f4-4376-9f93-0a0b3b595505 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:18.465897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de0983-0da5-420a-931f-82f60f1eff3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:18.466871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de0983-0da5-420a-931f-82f60f1eff3d admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:18.467261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de0983-0da5-420a-931f-82f60f1eff3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:18.467651 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de0983-0da5-420a-931f-82f60f1eff3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:18.468961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de0983-0da5-420a-931f-82f60f1eff3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:18.470853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de0983-0da5-420a-931f-82f60f1eff3d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:18.482316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1100/2229] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:51:18.487716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fab6b-f79d-4595-a64f-50c356df83de admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:18.499652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fab6b-f79d-4595-a64f-50c356df83de admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:18.505986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51edc7-f7a0-434a-b6b9-e9d894fe5482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:18.507198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51edc7-f7a0-434a-b6b9-e9d894fe5482 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:18.507642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51edc7-f7a0-434a-b6b9-e9d894fe5482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:18.508098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51edc7-f7a0-434a-b6b9-e9d894fe5482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:18.509133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51edc7-f7a0-434a-b6b9-e9d894fe5482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:18.511310 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51edc7-f7a0-434a-b6b9-e9d894fe5482 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:18.529054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1130/2230] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:51:18.529905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de0983-0da5-420a-931f-82f60f1eff3d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:18.541197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de0983-0da5-420a-931f-82f60f1eff3d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:18.549959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e6220-5021-43ef-8e9e-58adc4170b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:18.550789 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e6220-5021-43ef-8e9e-58adc4170b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:18.551185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e6220-5021-43ef-8e9e-58adc4170b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:18.551563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e6220-5021-43ef-8e9e-58adc4170b56 admin admin] PATH_INFO: `/v3/projects/547d1c9f06a54148bddee19d275c1685/users/5316bbdfb62d4db18be95cd52041b569/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:18.569905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e6220-5021-43ef-8e9e-58adc4170b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=547d1c9f06a54148bddee19d275c1685, user_id=5316bbdfb62d4db18be95cd52041b569, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:18.572724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e6220-5021-43ef-8e9e-58adc4170b56 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:19.008401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1101/2231] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:51:18 2022] PUT /identity/v3/projects/547d1c9f06a54148bddee19d275c1685/users/5316bbdfb62d4db18be95cd52041b569/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:51:19.019441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73721f-779e-40a3-880c-bbc6ceec5cf7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:19.019725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73721f-779e-40a3-880c-bbc6ceec5cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:19.020001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73721f-779e-40a3-880c-bbc6ceec5cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:19.049357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cd73721f-779e-40a3-880c-bbc6ceec5cf7 None None] MFA Rules not processed for user `5316bbdfb62d4db18be95cd52041b569`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:19.050646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cd73721f-779e-40a3-880c-bbc6ceec5cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:19.119943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1131/2232] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:19.132254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e6220-5021-43ef-8e9e-58adc4170b56 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:19.134076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-442e6220-5021-43ef-8e9e-58adc4170b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:19.224319 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e6220-5021-43ef-8e9e-58adc4170b56 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:19.228353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035784-a398-490b-899f-eed408ce3762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:19.229913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035784-a398-490b-899f-eed408ce3762 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:19.230220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035784-a398-490b-899f-eed408ce3762 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:19.230501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035784-a398-490b-899f-eed408ce3762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:19.246020 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2035784-a398-490b-899f-eed408ce3762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:19.247950 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2035784-a398-490b-899f-eed408ce3762 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:19.273862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1102/2233] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:51:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:51:21.652733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73721f-779e-40a3-880c-bbc6ceec5cf7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:21.654257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cd73721f-779e-40a3-880c-bbc6ceec5cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:21.873783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73721f-779e-40a3-880c-bbc6ceec5cf7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:21.893374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac50023-141d-4fb9-997b-199baf6b9366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:21.893374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac50023-141d-4fb9-997b-199baf6b9366 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:21.893374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac50023-141d-4fb9-997b-199baf6b9366 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:21.893374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac50023-141d-4fb9-997b-199baf6b9366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:21.893935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0ac50023-141d-4fb9-997b-199baf6b9366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:21.979181 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:21.996003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac50023-141d-4fb9-997b-199baf6b9366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:21.997614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac50023-141d-4fb9-997b-199baf6b9366 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:22.018034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1132/2234] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:51:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:51:22.704805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035784-a398-490b-899f-eed408ce3762 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:22.775226 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035784-a398-490b-899f-eed408ce3762 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:22.779635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:22.780263 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:22.781102 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:22.781449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] PATH_INFO: `/v3/users/5316bbdfb62d4db18be95cd52041b569` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:22.782599 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5316bbdfb62d4db18be95cd52041b569)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:22.784495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:22.816522 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:23.083155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5316bbdfb62d4db18be95cd52041b569'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:23.191777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5316bbdfb62d4db18be95cd52041b569'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:23.192939 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5316bbdfb62d4db18be95cd52041b569'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:23.208304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5316bbdfb62d4db18be95cd52041b569'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:23.248250 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5316bbdfb62d4db18be95cd52041b569'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:23.276921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1103/2235] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:51:22 2022] DELETE /identity/v3/users/5316bbdfb62d4db18be95cd52041b569 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:51:23.281391 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455302c-2367-41b2-ae25-abe2e49f1910 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:23.281949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455302c-2367-41b2-ae25-abe2e49f1910 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:23.281949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455302c-2367-41b2-ae25-abe2e49f1910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:23.323452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a455302c-2367-41b2-ae25-abe2e49f1910 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:23.325268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a455302c-2367-41b2-ae25-abe2e49f1910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:23.472157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1133/2236] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:23.486879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:23.488717 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:23.618297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e662b-e698-4997-b19f-ed8212580756 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:23.623181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7412c2-61c4-4ba7-af09-7a2edb066857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:23.624491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7412c2-61c4-4ba7-af09-7a2edb066857 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:23.625066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7412c2-61c4-4ba7-af09-7a2edb066857 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:23.625551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7412c2-61c4-4ba7-af09-7a2edb066857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:23.638827 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7412c2-61c4-4ba7-af09-7a2edb066857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:23.641554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7412c2-61c4-4ba7-af09-7a2edb066857 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:23.661837 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1104/2237] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:51:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:51:24.402854 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455302c-2367-41b2-ae25-abe2e49f1910 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:24.404409 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a455302c-2367-41b2-ae25-abe2e49f1910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:24.456526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455302c-2367-41b2-ae25-abe2e49f1910 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:24.484666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33453844-55c2-4d82-a71e-31305dbdd07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:24.490778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33453844-55c2-4d82-a71e-31305dbdd07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:24.491284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33453844-55c2-4d82-a71e-31305dbdd07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:24.491727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33453844-55c2-4d82-a71e-31305dbdd07b admin admin] PATH_INFO: `/v3/projects/547d1c9f06a54148bddee19d275c1685` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:24.495700 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33453844-55c2-4d82-a71e-31305dbdd07b admin admin] RBAC: Authorizing `identity:delete_project(project_id=547d1c9f06a54148bddee19d275c1685)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:24.501974 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33453844-55c2-4d82-a71e-31305dbdd07b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:24.666452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-33453844-55c2-4d82-a71e-31305dbdd07b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '547d1c9f06a54148bddee19d275c1685', 'request_id': 'req-33453844-55c2-4d82-a71e-31305dbdd07b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:24.681190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-33453844-55c2-4d82-a71e-31305dbdd07b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '547d1c9f06a54148bddee19d275c1685', 'request_id': 'req-33453844-55c2-4d82-a71e-31305dbdd07b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:24.690820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-33453844-55c2-4d82-a71e-31305dbdd07b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '547d1c9f06a54148bddee19d275c1685', 'request_id': 'req-33453844-55c2-4d82-a71e-31305dbdd07b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:24.766369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-33453844-55c2-4d82-a71e-31305dbdd07b admin admin] The token cache is being invalidate because project 547d1c9f06a54148bddee19d275c1685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:51:24.766962 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-33453844-55c2-4d82-a71e-31305dbdd07b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:24.767811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-33453844-55c2-4d82-a71e-31305dbdd07b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:24.773898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1134/2238] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:51:24 2022] DELETE /identity/v3/projects/547d1c9f06a54148bddee19d275c1685 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:51:24.774772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55024717-fc54-41f0-ab77-10a6efb78217 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:24.775241 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55024717-fc54-41f0-ab77-10a6efb78217 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:24.775353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55024717-fc54-41f0-ab77-10a6efb78217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:24.815613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-55024717-fc54-41f0-ab77-10a6efb78217 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:24.816900 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-55024717-fc54-41f0-ab77-10a6efb78217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:24.971882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1105/2239] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:24.980932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57a1ac-232b-49ed-bc19-031e51e09c90 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:24.981222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57a1ac-232b-49ed-bc19-031e51e09c90 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:24.981501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57a1ac-232b-49ed-bc19-031e51e09c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:25.018036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bf57a1ac-232b-49ed-bc19-031e51e09c90 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:25.020526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bf57a1ac-232b-49ed-bc19-031e51e09c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:25.032011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1135/2240] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:25.038513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55024717-fc54-41f0-ab77-10a6efb78217 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:25.056017 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55024717-fc54-41f0-ab77-10a6efb78217 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:25.062486 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51f2a7-0172-46cc-b333-33266ad142f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:25.066842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51f2a7-0172-46cc-b333-33266ad142f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:25.066842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51f2a7-0172-46cc-b333-33266ad142f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:25.066842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51f2a7-0172-46cc-b333-33266ad142f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:25.066842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51f2a7-0172-46cc-b333-33266ad142f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:51:25.066842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51f2a7-0172-46cc-b333-33266ad142f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:25.068152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51f2a7-0172-46cc-b333-33266ad142f8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:25.091971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1106/2241] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:51:25.094122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57a1ac-232b-49ed-bc19-031e51e09c90 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:25.105172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57a1ac-232b-49ed-bc19-031e51e09c90 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:25.111117 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7954cf3-35b6-47d8-8fe0-087e2d8df320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:25.111965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7954cf3-35b6-47d8-8fe0-087e2d8df320 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:25.112536 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7954cf3-35b6-47d8-8fe0-087e2d8df320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:25.113190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7954cf3-35b6-47d8-8fe0-087e2d8df320 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:25.114177 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7954cf3-35b6-47d8-8fe0-087e2d8df320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:25.117514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7954cf3-35b6-47d8-8fe0-087e2d8df320 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:25.287384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1136/2242] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:51:25 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:51:25.288889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51f2a7-0172-46cc-b333-33266ad142f8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:25.302746 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51f2a7-0172-46cc-b333-33266ad142f8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:25.380049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:25.463378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e86af-e35b-43e5-8633-dbb4d01a30e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:25.464238 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e86af-e35b-43e5-8633-dbb4d01a30e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:25.466689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e86af-e35b-43e5-8633-dbb4d01a30e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:25.467419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e86af-e35b-43e5-8633-dbb4d01a30e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:25.472751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e86af-e35b-43e5-8633-dbb4d01a30e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:25.474951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e86af-e35b-43e5-8633-dbb4d01a30e6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:25.665292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1107/2243] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:51:25 2022] POST /identity/v3/users => generated 288 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:51:25.666656 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7954cf3-35b6-47d8-8fe0-087e2d8df320 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:25.690622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7954cf3-35b6-47d8-8fe0-087e2d8df320 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:25.704356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295c618-5989-41ea-8670-267cbe54ba28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:25.705224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295c618-5989-41ea-8670-267cbe54ba28 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:25.705224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295c618-5989-41ea-8670-267cbe54ba28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:25.705396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295c618-5989-41ea-8670-267cbe54ba28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:25.706411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295c618-5989-41ea-8670-267cbe54ba28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:25.710425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295c618-5989-41ea-8670-267cbe54ba28 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:25.734927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1137/2244] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:51:25.736396 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e86af-e35b-43e5-8633-dbb4d01a30e6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:25.750169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e86af-e35b-43e5-8633-dbb4d01a30e6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:25.756531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65b21a-a946-4860-9e80-defe5ac4ed43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:25.757358 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65b21a-a946-4860-9e80-defe5ac4ed43 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:25.757679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65b21a-a946-4860-9e80-defe5ac4ed43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:25.758003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65b21a-a946-4860-9e80-defe5ac4ed43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:25.760011 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65b21a-a946-4860-9e80-defe5ac4ed43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:25.761975 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65b21a-a946-4860-9e80-defe5ac4ed43 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:25.779984 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1108/2245] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:51:25.782132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295c618-5989-41ea-8670-267cbe54ba28 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:25.798598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295c618-5989-41ea-8670-267cbe54ba28 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:25.805419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12c960-bb0e-4126-bb44-b148d38acb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:25.806328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12c960-bb0e-4126-bb44-b148d38acb80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:25.806614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12c960-bb0e-4126-bb44-b148d38acb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:25.806892 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12c960-bb0e-4126-bb44-b148d38acb80 admin admin] PATH_INFO: `/v3/projects/0479f090a32d4032aa22787110ed03f5/users/defabe33a21c47078c2e959211bc7a82/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:25.844264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12c960-bb0e-4126-bb44-b148d38acb80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0479f090a32d4032aa22787110ed03f5, user_id=defabe33a21c47078c2e959211bc7a82, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:25.847090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12c960-bb0e-4126-bb44-b148d38acb80 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:25.950040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1138/2246] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:51:25 2022] PUT /identity/v3/projects/0479f090a32d4032aa22787110ed03f5/users/defabe33a21c47078c2e959211bc7a82/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:51:25.961553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de9fe8-e6fd-4180-9549-47648738e427 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:25.961842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de9fe8-e6fd-4180-9549-47648738e427 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:25.962118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de9fe8-e6fd-4180-9549-47648738e427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:26.028341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-95de9fe8-e6fd-4180-9549-47648738e427 None None] MFA Rules not processed for user `defabe33a21c47078c2e959211bc7a82`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:26.029608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-95de9fe8-e6fd-4180-9549-47648738e427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:26.206041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1109/2247] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:51:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:26.223811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12c960-bb0e-4126-bb44-b148d38acb80 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:26.225487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-9b12c960-bb0e-4126-bb44-b148d38acb80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:26.247506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:26.316568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12c960-bb0e-4126-bb44-b148d38acb80 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:26.322463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82464b-ccaf-48b2-b481-067948911ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:26.323543 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82464b-ccaf-48b2-b481-067948911ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:26.323995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82464b-ccaf-48b2-b481-067948911ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:26.324443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82464b-ccaf-48b2-b481-067948911ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:26.335145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82464b-ccaf-48b2-b481-067948911ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:26.338197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82464b-ccaf-48b2-b481-067948911ab4 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:26.362197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1139/2248] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:51:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:51:34.319201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02bf33-a2f1-42fa-9416-45f7a5a29fea None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:34.319758 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02bf33-a2f1-42fa-9416-45f7a5a29fea None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:34.319869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02bf33-a2f1-42fa-9416-45f7a5a29fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:34.338822 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ef02bf33-a2f1-42fa-9416-45f7a5a29fea None None] MFA Rules not processed for user `04da0f4276d6457c889ecb3006842e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:34.340043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ef02bf33-a2f1-42fa-9416-45f7a5a29fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:34.403687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1110/2249] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:51:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:35.833992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b469f-e907-41a9-b56a-49791453c8a0 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:35.834601 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b469f-e907-41a9-b56a-49791453c8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:35.834718 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b469f-e907-41a9-b56a-49791453c8a0 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:35.836039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1140/2250] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:51:35.842235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495e40e-e8f0-40fa-b32b-0c4826ff4925 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:35.842766 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495e40e-e8f0-40fa-b32b-0c4826ff4925 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:35.842976 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495e40e-e8f0-40fa-b32b-0c4826ff4925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:35.862870 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a495e40e-e8f0-40fa-b32b-0c4826ff4925 None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:35.864085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a495e40e-e8f0-40fa-b32b-0c4826ff4925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:35.918334 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:35.949255 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1111/2251] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:51:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:35.962219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b469f-e907-41a9-b56a-49791453c8a0 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:35.963986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-954b469f-e907-41a9-b56a-49791453c8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:36.024534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b469f-e907-41a9-b56a-49791453c8a0 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:36.028162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee53f6-31e9-4288-b8b7-05fbca0b2050 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a58f9101f2d4785b96f0cc55be4330b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:36.028769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee53f6-31e9-4288-b8b7-05fbca0b2050 service glance] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:36.029050 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee53f6-31e9-4288-b8b7-05fbca0b2050 service glance] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:36.029329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee53f6-31e9-4288-b8b7-05fbca0b2050 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:36.039765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee53f6-31e9-4288-b8b7-05fbca0b2050 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:36.041230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee53f6-31e9-4288-b8b7-05fbca0b2050 service glance] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:36.052933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1141/2252] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:51:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:51:36.276990 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b830f8-e11c-422c-bb4a-77e81e36ff41 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:36.284270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b830f8-e11c-422c-bb4a-77e81e36ff41 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:36.284270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b830f8-e11c-422c-bb4a-77e81e36ff41 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:36.284270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1112/2253] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:51:36.285460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546d2fa-adc0-40cb-878c-305efa6a0918 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:36.285802 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546d2fa-adc0-40cb-878c-305efa6a0918 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:36.286142 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546d2fa-adc0-40cb-878c-305efa6a0918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:36.313065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6546d2fa-adc0-40cb-878c-305efa6a0918 None None] MFA Rules not processed for user `88a7cdd610f74330a5da69c0de8472db`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:36.315554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6546d2fa-adc0-40cb-878c-305efa6a0918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:36.372189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1142/2254] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:51:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:51:36.385772 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b830f8-e11c-422c-bb4a-77e81e36ff41 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:36.388248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-14b830f8-e11c-422c-bb4a-77e81e36ff41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:36.402478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b830f8-e11c-422c-bb4a-77e81e36ff41 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:36.406693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c626b5-6f74-4962-9e4a-423c2ea990d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:36.407377 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c626b5-6f74-4962-9e4a-423c2ea990d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:36.407733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c626b5-6f74-4962-9e4a-423c2ea990d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:36.408075 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c626b5-6f74-4962-9e4a-423c2ea990d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:36.415727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c626b5-6f74-4962-9e4a-423c2ea990d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:36.419105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c626b5-6f74-4962-9e4a-423c2ea990d9 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:36.426012 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1113/2255] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:51:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:51:36.557922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5724a2-7787-4a6d-b41b-c3db2f770249 None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:36.561102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5724a2-7787-4a6d-b41b-c3db2f770249 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:36.561102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5724a2-7787-4a6d-b41b-c3db2f770249 None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:36.561102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1143/2256] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:51:36.565845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea5b74-92c0-4fd9-be4a-2875dbaa86be None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:36.566134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea5b74-92c0-4fd9-be4a-2875dbaa86be None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:36.566410 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea5b74-92c0-4fd9-be4a-2875dbaa86be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:36.587916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-95ea5b74-92c0-4fd9-be4a-2875dbaa86be None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:36.589190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-95ea5b74-92c0-4fd9-be4a-2875dbaa86be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:36.600411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1114/2257] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:51:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:51:36.614020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5724a2-7787-4a6d-b41b-c3db2f770249 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:36.615804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2e5724a2-7787-4a6d-b41b-c3db2f770249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:36.628937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5724a2-7787-4a6d-b41b-c3db2f770249 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:36.632557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be7876-6199-4017-9b85-514b9c6e8794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:36.633145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be7876-6199-4017-9b85-514b9c6e8794 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:36.633422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be7876-6199-4017-9b85-514b9c6e8794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:36.633721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be7876-6199-4017-9b85-514b9c6e8794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:36.644007 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be7876-6199-4017-9b85-514b9c6e8794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:36.645468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be7876-6199-4017-9b85-514b9c6e8794 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:36.657801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1144/2258] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:51:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:51:38.348286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea5b74-92c0-4fd9-be4a-2875dbaa86be None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:38.412977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea5b74-92c0-4fd9-be4a-2875dbaa86be None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:38.418195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:38.419138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:38.419138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:38.419341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] PATH_INFO: `/v3/users/defabe33a21c47078c2e959211bc7a82` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:38.420457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=defabe33a21c47078c2e959211bc7a82)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:38.422409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:38.653367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'defabe33a21c47078c2e959211bc7a82'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:38.737169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'defabe33a21c47078c2e959211bc7a82'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:38.738160 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'defabe33a21c47078c2e959211bc7a82'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:38.760540 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'defabe33a21c47078c2e959211bc7a82'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:38.776551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'defabe33a21c47078c2e959211bc7a82'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:38.783790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1115/2259] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:51:38 2022] DELETE /identity/v3/users/defabe33a21c47078c2e959211bc7a82 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:51:38.788055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c7bf-9b66-436f-af48-30a6dba61e9f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:38.788555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c7bf-9b66-436f-af48-30a6dba61e9f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:38.788685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c7bf-9b66-436f-af48-30a6dba61e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:38.809412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7042c7bf-9b66-436f-af48-30a6dba61e9f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:38.810684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7042c7bf-9b66-436f-af48-30a6dba61e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:38.874961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:38.897647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1145/2260] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:38.904422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:38.906795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:38.948271 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:38.975648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298fa9f-4e13-4385-84b9-052d6e5d2c4c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:38.979927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8889d-32ef-4b52-8ce4-0dcf11d3f620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:38.980622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8889d-32ef-4b52-8ce4-0dcf11d3f620 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:38.980913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8889d-32ef-4b52-8ce4-0dcf11d3f620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:38.981186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8889d-32ef-4b52-8ce4-0dcf11d3f620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:38.998419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8889d-32ef-4b52-8ce4-0dcf11d3f620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:39.000491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8889d-32ef-4b52-8ce4-0dcf11d3f620 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:39.012577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1116/2261] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:51:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:51:39.873810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042c7bf-9b66-436f-af48-30a6dba61e9f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:39.875482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7042c7bf-9b66-436f-af48-30a6dba61e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:39.888865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042c7bf-9b66-436f-af48-30a6dba61e9f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:39.900915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:39.900915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:39.900915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:39.900915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] PATH_INFO: `/v3/projects/0479f090a32d4032aa22787110ed03f5` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:39.901677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0479f090a32d4032aa22787110ed03f5)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:39.901677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:40.048014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0479f090a32d4032aa22787110ed03f5', 'request_id': 'req-c08652e7-c5f3-479b-beec-1e5f8c7220fb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:40.055754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0479f090a32d4032aa22787110ed03f5', 'request_id': 'req-c08652e7-c5f3-479b-beec-1e5f8c7220fb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:40.064469 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0479f090a32d4032aa22787110ed03f5', 'request_id': 'req-c08652e7-c5f3-479b-beec-1e5f8c7220fb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:40.188182 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] The token cache is being invalidate because project 0479f090a32d4032aa22787110ed03f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:51:40.188529 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:40.190029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:51:40.192286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1146/2262] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:51:39 2022] DELETE /identity/v3/projects/0479f090a32d4032aa22787110ed03f5 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:51:41.760318 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f910a-19de-4e68-b636-49c660841140 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:41.760869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f910a-19de-4e68-b636-49c660841140 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:41.760869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f910a-19de-4e68-b636-49c660841140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:41.782092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-326f910a-19de-4e68-b636-49c660841140 None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:51:41.783423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-326f910a-19de-4e68-b636-49c660841140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:41.843108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1117/2263] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:51:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:51:41.856035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:41.857539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:41.932599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08652e7-c5f3-479b-beec-1e5f8c7220fb admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:41.938368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f4954-e8f5-4dee-a243-f226ab8586e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:41.941106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f4954-e8f5-4dee-a243-f226ab8586e7 service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:41.941106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f4954-e8f5-4dee-a243-f226ab8586e7 service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:41.941106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f4954-e8f5-4dee-a243-f226ab8586e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:41.947901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f4954-e8f5-4dee-a243-f226ab8586e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:41.949489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f4954-e8f5-4dee-a243-f226ab8586e7 service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:41.958515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1147/2264] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:51:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:51:51.836223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=36 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:51.955076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:51:56.685243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f910a-19de-4e68-b636-49c660841140 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:51:56.685243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-326f910a-19de-4e68-b636-49c660841140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:56.758521 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f910a-19de-4e68-b636-49c660841140 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:51:56.763970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45899871-7425-4858-818b-9c5db79f6759 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0aa6b215f646c280af245a89ccf849', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:51:56.764567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45899871-7425-4858-818b-9c5db79f6759 service placement] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:51:56.764845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45899871-7425-4858-818b-9c5db79f6759 service placement] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:51:56.765112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45899871-7425-4858-818b-9c5db79f6759 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:51:56.767639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-45899871-7425-4858-818b-9c5db79f6759 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:51:56.779085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45899871-7425-4858-818b-9c5db79f6759 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:51:56.780664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45899871-7425-4858-818b-9c5db79f6759 service placement] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:51:56.790828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1118/2265] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:51:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:52:06.002664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bc7f0-c8ab-4e97-9d23-6be6275d7688 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:52:06.003204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bc7f0-c8ab-4e97-9d23-6be6275d7688 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:52:06.003204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bc7f0-c8ab-4e97-9d23-6be6275d7688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:52:06.022859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4c4bc7f0-c8ab-4e97-9d23-6be6275d7688 None None] MFA Rules not processed for user `00491255f4b44125b6dc9f816c9bbb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:52:06.024099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4c4bc7f0-c8ab-4e97-9d23-6be6275d7688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:52:06.087223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1148/2266] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:52:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:52:06.786409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=32 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:52:06.845502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf754ac-1e3f-4aaf-947d-9b3ecba2c22c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:52:06.845824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf754ac-1e3f-4aaf-947d-9b3ecba2c22c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:52:06.846094 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf754ac-1e3f-4aaf-947d-9b3ecba2c22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:52:06.866457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-eaf754ac-1e3f-4aaf-947d-9b3ecba2c22c None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:52:06.867668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-eaf754ac-1e3f-4aaf-947d-9b3ecba2c22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:52:06.874840 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1119/2267] 10.222.0.57 () {58 vars in 967 bytes} [Tue Aug 9 18:52:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:52:06.887932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bc7f0-c8ab-4e97-9d23-6be6275d7688 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:52:06.900348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bc7f0-c8ab-4e97-9d23-6be6275d7688 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:52:06.905903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb76b5b-9411-4065-aa81-b770ad3fa97f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:52:06.906701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb76b5b-9411-4065-aa81-b770ad3fa97f service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:52:06.907490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb76b5b-9411-4065-aa81-b770ad3fa97f service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:52:06.907757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb76b5b-9411-4065-aa81-b770ad3fa97f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:52:06.916432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb76b5b-9411-4065-aa81-b770ad3fa97f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:52:06.918050 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb76b5b-9411-4065-aa81-b770ad3fa97f service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:52:06.926560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1149/2268] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:52:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:52:16.861471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:52:16.922971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=31 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:53:02.567327 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a125a5-2755-40ce-9423-8eb0318ecfff None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:53:02.567925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a125a5-2755-40ce-9423-8eb0318ecfff None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:53:02.567925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a125a5-2755-40ce-9423-8eb0318ecfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:53:02.586909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-75a125a5-2755-40ce-9423-8eb0318ecfff None None] MFA Rules not processed for user `ef8af9fd9e844ca4be10caa880357a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:53:02.588473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-75a125a5-2755-40ce-9423-8eb0318ecfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:53:02.643219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1120/2269] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:53:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:53:07.826346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029af57-0600-4cfc-962c-89c2f6f5bd1f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:53:07.826908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029af57-0600-4cfc-962c-89c2f6f5bd1f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:53:07.826908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029af57-0600-4cfc-962c-89c2f6f5bd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:53:07.846826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-6029af57-0600-4cfc-962c-89c2f6f5bd1f None None] MFA Rules not processed for user `04da0f4276d6457c889ecb3006842e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:53:07.848034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6029af57-0600-4cfc-962c-89c2f6f5bd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:53:07.908666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1150/2270] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:53:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:53:12.636500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=25 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:53:17.901684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=25 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:53:27.138977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d165fd-5363-4e9b-afbd-f0d7732789bd None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:53:27.138977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d165fd-5363-4e9b-afbd-f0d7732789bd None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:53:27.139583 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d165fd-5363-4e9b-afbd-f0d7732789bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:53:27.157825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-14d165fd-5363-4e9b-afbd-f0d7732789bd None None] MFA Rules not processed for user `a6e48ee30c61424ba0c82ac4fc5cd06c`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:53:27.158955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-14d165fd-5363-4e9b-afbd-f0d7732789bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:53:27.212667 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1121/2271] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:53:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:53:27.226257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7672d8-53d7-4bb6-84c5-9c866093eaee None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:53:27.226257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7672d8-53d7-4bb6-84c5-9c866093eaee None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:53:27.226257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7672d8-53d7-4bb6-84c5-9c866093eaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:53:27.246905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-3c7672d8-53d7-4bb6-84c5-9c866093eaee None None] MFA Rules not processed for user `00491255f4b44125b6dc9f816c9bbb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:53:27.248171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3c7672d8-53d7-4bb6-84c5-9c866093eaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:53:27.255614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1151/2272] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:53:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:53:37.206139 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=25 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:53:37.240846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:53:46.101251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d165fd-5363-4e9b-afbd-f0d7732789bd None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:53:46.102707 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-14d165fd-5363-4e9b-afbd-f0d7732789bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:53:46.159693 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d165fd-5363-4e9b-afbd-f0d7732789bd None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:53:46.163252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616aa5e-c8d6-4c97-b7d6-c7f33851649d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:53:46.163852 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616aa5e-c8d6-4c97-b7d6-c7f33851649d service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:53:46.164129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616aa5e-c8d6-4c97-b7d6-c7f33851649d service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:53:46.164395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616aa5e-c8d6-4c97-b7d6-c7f33851649d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:53:46.166634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3616aa5e-c8d6-4c97-b7d6-c7f33851649d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:53:46.235864 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616aa5e-c8d6-4c97-b7d6-c7f33851649d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:53:46.237288 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616aa5e-c8d6-4c97-b7d6-c7f33851649d service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:53:46.248488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1122/2273] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:53:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:53:49.772594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7672d8-53d7-4bb6-84c5-9c866093eaee None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:53:49.774057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3c7672d8-53d7-4bb6-84c5-9c866093eaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:53:49.786600 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7672d8-53d7-4bb6-84c5-9c866093eaee None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:53:49.790233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415a2b1-065a-4758-a2ac-deef76689b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:53:49.790834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415a2b1-065a-4758-a2ac-deef76689b8d service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:53:49.791139 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415a2b1-065a-4758-a2ac-deef76689b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:53:49.791416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415a2b1-065a-4758-a2ac-deef76689b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:53:49.793178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6415a2b1-065a-4758-a2ac-deef76689b8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:53:49.808458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415a2b1-065a-4758-a2ac-deef76689b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:53:49.809956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415a2b1-065a-4758-a2ac-deef76689b8d service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:53:49.821369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1152/2274] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:53:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:53:51.615042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd742b50-72e1-4d66-a5ea-6499004795d1 None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:53:51.615600 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd742b50-72e1-4d66-a5ea-6499004795d1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:53:51.615600 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd742b50-72e1-4d66-a5ea-6499004795d1 None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:53:51.617116 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1123/2275] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:53:51.623059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad4578-5d4e-459d-9565-a59ef1044f94 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:53:51.623703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad4578-5d4e-459d-9565-a59ef1044f94 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:53:51.623703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad4578-5d4e-459d-9565-a59ef1044f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:53:51.644773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-08ad4578-5d4e-459d-9565-a59ef1044f94 None None] MFA Rules not processed for user `88a7cdd610f74330a5da69c0de8472db`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:53:51.646042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-08ad4578-5d4e-459d-9565-a59ef1044f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:53:51.653913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1153/2276] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:53:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:53:51.667780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd742b50-72e1-4d66-a5ea-6499004795d1 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:53:51.680678 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd742b50-72e1-4d66-a5ea-6499004795d1 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:53:51.686125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c71d4-6470-4632-9fad-b9fd02e00b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:53:51.686783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c71d4-6470-4632-9fad-b9fd02e00b59 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:53:51.687065 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c71d4-6470-4632-9fad-b9fd02e00b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:53:51.687339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c71d4-6470-4632-9fad-b9fd02e00b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:53:51.695398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c71d4-6470-4632-9fad-b9fd02e00b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:53:51.696934 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c71d4-6470-4632-9fad-b9fd02e00b59 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:53:51.717387 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1124/2277] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:53:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:53:51.802092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027502c-5926-4ce4-a78c-ec5a7213bd7d None None] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:53:51.802092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027502c-5926-4ce4-a78c-ec5a7213bd7d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:53:51.802092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027502c-5926-4ce4-a78c-ec5a7213bd7d None None] PATH_INFO: `/` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:53:51.804769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1154/2278] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:53:51.810607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbb541-3813-4d95-8111-a4e208795423 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:53:51.811591 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbb541-3813-4d95-8111-a4e208795423 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:53:51.812011 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbb541-3813-4d95-8111-a4e208795423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:53:51.842998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c4bbb541-3813-4d95-8111-a4e208795423 None None] MFA Rules not processed for user `e2ecd7d1cce644b0982cc23740537933`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:53:51.844198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c4bbb541-3813-4d95-8111-a4e208795423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:53:51.853283 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1125/2279] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:53:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:53:51.864432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027502c-5926-4ce4-a78c-ec5a7213bd7d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:53:51.875414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027502c-5926-4ce4-a78c-ec5a7213bd7d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:53:51.879026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707bba9-fc2f-4246-84f0-f5d54f92fbbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:53:51.879647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707bba9-fc2f-4246-84f0-f5d54f92fbbe service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:53:51.879930 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707bba9-fc2f-4246-84f0-f5d54f92fbbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:53:51.880203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707bba9-fc2f-4246-84f0-f5d54f92fbbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:53:51.890505 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707bba9-fc2f-4246-84f0-f5d54f92fbbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:53:51.891973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707bba9-fc2f-4246-84f0-f5d54f92fbbe service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:53:51.903647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1155/2280] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:53:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:01.341617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbb541-3813-4d95-8111-a4e208795423 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:01.341617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c4bbb541-3813-4d95-8111-a4e208795423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:01.354632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbb541-3813-4d95-8111-a4e208795423 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:01.360119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa623af5-1ba1-431b-a690-076edc155613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:01.360719 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa623af5-1ba1-431b-a690-076edc155613 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:01.360997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa623af5-1ba1-431b-a690-076edc155613 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:01.361267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa623af5-1ba1-431b-a690-076edc155613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:01.363003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-aa623af5-1ba1-431b-a690-076edc155613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:01.375334 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa623af5-1ba1-431b-a690-076edc155613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:01.376898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa623af5-1ba1-431b-a690-076edc155613 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:01.386302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1126/2281] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:01.898276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=15 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:06.610763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707bba9-fc2f-4246-84f0-f5d54f92fbbe service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:06.612188 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7707bba9-fc2f-4246-84f0-f5d54f92fbbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:06.626376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707bba9-fc2f-4246-84f0-f5d54f92fbbe service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:06.631719 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47703ba-5da4-4998-8ccf-7fa308c26714 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:06.632304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47703ba-5da4-4998-8ccf-7fa308c26714 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:06.632575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47703ba-5da4-4998-8ccf-7fa308c26714 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:06.632845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47703ba-5da4-4998-8ccf-7fa308c26714 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:06.634532 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b47703ba-5da4-4998-8ccf-7fa308c26714 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:06.644456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47703ba-5da4-4998-8ccf-7fa308c26714 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:06.646045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47703ba-5da4-4998-8ccf-7fa308c26714 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:06.655419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1156/2282] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:11.381712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=69 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:13.582734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa623af5-1ba1-431b-a690-076edc155613 service nova] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:13.593535 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa623af5-1ba1-431b-a690-076edc155613 service nova] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:13.597319 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f1d1c-84e3-4f5c-9766-9bda2ea89376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:13.597966 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f1d1c-84e3-4f5c-9766-9bda2ea89376 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:13.598222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f1d1c-84e3-4f5c-9766-9bda2ea89376 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:13.598491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f1d1c-84e3-4f5c-9766-9bda2ea89376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:13.600381 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f46f1d1c-84e3-4f5c-9766-9bda2ea89376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:13.611289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f1d1c-84e3-4f5c-9766-9bda2ea89376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:13.612715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f1d1c-84e3-4f5c-9766-9bda2ea89376 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:13.621295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1127/2283] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:16.156095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47703ba-5da4-4998-8ccf-7fa308c26714 service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:16.165321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47703ba-5da4-4998-8ccf-7fa308c26714 service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:16.168873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be68443-19e8-4b25-a593-a2de8f777e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:16.169459 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be68443-19e8-4b25-a593-a2de8f777e35 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:16.169810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be68443-19e8-4b25-a593-a2de8f777e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:16.170082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be68443-19e8-4b25-a593-a2de8f777e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:16.171812 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0be68443-19e8-4b25-a593-a2de8f777e35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:16.239750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be68443-19e8-4b25-a593-a2de8f777e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:16.241197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be68443-19e8-4b25-a593-a2de8f777e35 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:16.252875 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1157/2284] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:20.118513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f1d1c-84e3-4f5c-9766-9bda2ea89376 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:20.119978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f46f1d1c-84e3-4f5c-9766-9bda2ea89376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:20.134444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f1d1c-84e3-4f5c-9766-9bda2ea89376 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:20.139207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018e739-3625-4f93-8783-a789cd0423ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:20.139840 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018e739-3625-4f93-8783-a789cd0423ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:20.140121 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018e739-3625-4f93-8783-a789cd0423ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:20.140394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018e739-3625-4f93-8783-a789cd0423ac admin admin] PATH_INFO: `/v3/users/04da0f4276d6457c889ecb3006842e9c` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:20.141796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018e739-3625-4f93-8783-a789cd0423ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=04da0f4276d6457c889ecb3006842e9c)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:20.143795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018e739-3625-4f93-8783-a789cd0423ac admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:20.351977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c018e739-3625-4f93-8783-a789cd0423ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04da0f4276d6457c889ecb3006842e9c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:20.440115 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c018e739-3625-4f93-8783-a789cd0423ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04da0f4276d6457c889ecb3006842e9c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:20.440668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c018e739-3625-4f93-8783-a789cd0423ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04da0f4276d6457c889ecb3006842e9c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:20.449256 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c018e739-3625-4f93-8783-a789cd0423ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04da0f4276d6457c889ecb3006842e9c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:20.460884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c018e739-3625-4f93-8783-a789cd0423ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04da0f4276d6457c889ecb3006842e9c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:20.467455 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1128/2285] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:54:20 2022] DELETE /identity/v3/users/04da0f4276d6457c889ecb3006842e9c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:20.761505 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be68443-19e8-4b25-a593-a2de8f777e35 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:20.772200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be68443-19e8-4b25-a593-a2de8f777e35 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:20.832002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f03f64-0958-4349-a602-8c91ed6cf25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:20.832634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f03f64-0958-4349-a602-8c91ed6cf25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:20.832914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f03f64-0958-4349-a602-8c91ed6cf25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:20.833188 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f03f64-0958-4349-a602-8c91ed6cf25a admin admin] PATH_INFO: `/v3/projects/d3c7915d78334e95a41a32f0f3b5b084` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:20.834496 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f03f64-0958-4349-a602-8c91ed6cf25a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3c7915d78334e95a41a32f0f3b5b084)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:20.836421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f03f64-0958-4349-a602-8c91ed6cf25a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:20.969929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c4f03f64-0958-4349-a602-8c91ed6cf25a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3c7915d78334e95a41a32f0f3b5b084', 'request_id': 'req-c4f03f64-0958-4349-a602-8c91ed6cf25a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:20.978570 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c4f03f64-0958-4349-a602-8c91ed6cf25a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3c7915d78334e95a41a32f0f3b5b084', 'request_id': 'req-c4f03f64-0958-4349-a602-8c91ed6cf25a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:20.985293 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c4f03f64-0958-4349-a602-8c91ed6cf25a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3c7915d78334e95a41a32f0f3b5b084', 'request_id': 'req-c4f03f64-0958-4349-a602-8c91ed6cf25a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:21.067384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c4f03f64-0958-4349-a602-8c91ed6cf25a admin admin] The token cache is being invalidate because project d3c7915d78334e95a41a32f0f3b5b084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:54:21.067675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c4f03f64-0958-4349-a602-8c91ed6cf25a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:21.068333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c4f03f64-0958-4349-a602-8c91ed6cf25a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:21.069897 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1158/2286] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:54:20 2022] DELETE /identity/v3/projects/d3c7915d78334e95a41a32f0f3b5b084 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:54:21.075473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681f36b-86f8-4bc4-b6f3-412ab5324dce None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:21.075757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681f36b-86f8-4bc4-b6f3-412ab5324dce None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:21.076028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681f36b-86f8-4bc4-b6f3-412ab5324dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:21.096161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8681f36b-86f8-4bc4-b6f3-412ab5324dce None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:21.097397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8681f36b-86f8-4bc4-b6f3-412ab5324dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:21.155708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1129/2287] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:21.163511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d163f1-f740-4f4b-835c-8281407ee799 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:21.163795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d163f1-f740-4f4b-835c-8281407ee799 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:21.164071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d163f1-f740-4f4b-835c-8281407ee799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:21.184731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-17d163f1-f740-4f4b-835c-8281407ee799 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:21.186015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-17d163f1-f740-4f4b-835c-8281407ee799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:21.194345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1159/2288] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:21.199119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681f36b-86f8-4bc4-b6f3-412ab5324dce None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:21.209161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681f36b-86f8-4bc4-b6f3-412ab5324dce None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:21.214518 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d42be4-8d1d-4845-9550-1ee7be2cbb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:21.215118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d42be4-8d1d-4845-9550-1ee7be2cbb2e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:21.215395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d42be4-8d1d-4845-9550-1ee7be2cbb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:21.215668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d42be4-8d1d-4845-9550-1ee7be2cbb2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:21.216314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d42be4-8d1d-4845-9550-1ee7be2cbb2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:54:21.216618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d42be4-8d1d-4845-9550-1ee7be2cbb2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:21.218238 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d42be4-8d1d-4845-9550-1ee7be2cbb2e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:21.229980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1130/2289] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:54:21.234403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d163f1-f740-4f4b-835c-8281407ee799 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:21.239995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:21.243907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d163f1-f740-4f4b-835c-8281407ee799 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:21.248731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730b7cb-a69d-4289-9743-cc8e1423b8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:21.254064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730b7cb-a69d-4289-9743-cc8e1423b8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:21.254064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730b7cb-a69d-4289-9743-cc8e1423b8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:21.254064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730b7cb-a69d-4289-9743-cc8e1423b8d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:21.254064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730b7cb-a69d-4289-9743-cc8e1423b8d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:21.254064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730b7cb-a69d-4289-9743-cc8e1423b8d3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:21.374178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1160/2290] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:54:21 2022] POST /identity/v3/projects => generated 405 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:54:21.378327 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d42be4-8d1d-4845-9550-1ee7be2cbb2e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:21.388579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d42be4-8d1d-4845-9550-1ee7be2cbb2e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:21.431956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:21.444147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34e21c-42a0-48a2-a5cb-9ce609e10834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:21.444743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34e21c-42a0-48a2-a5cb-9ce609e10834 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:21.445023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34e21c-42a0-48a2-a5cb-9ce609e10834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:21.445297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34e21c-42a0-48a2-a5cb-9ce609e10834 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:21.445957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34e21c-42a0-48a2-a5cb-9ce609e10834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:21.447855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34e21c-42a0-48a2-a5cb-9ce609e10834 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:21.537284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1131/2291] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:54:21 2022] POST /identity/v3/users => generated 303 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:54:21.541109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730b7cb-a69d-4289-9743-cc8e1423b8d3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:21.551564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730b7cb-a69d-4289-9743-cc8e1423b8d3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:21.556660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e84080-e29e-4cc9-ae46-888c8f0ed622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:21.557256 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e84080-e29e-4cc9-ae46-888c8f0ed622 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:21.558073 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e84080-e29e-4cc9-ae46-888c8f0ed622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:21.558377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e84080-e29e-4cc9-ae46-888c8f0ed622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:21.559013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e84080-e29e-4cc9-ae46-888c8f0ed622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:21.560642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e84080-e29e-4cc9-ae46-888c8f0ed622 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:21.570908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1161/2292] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:54:21.575515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34e21c-42a0-48a2-a5cb-9ce609e10834 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:21.585824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34e21c-42a0-48a2-a5cb-9ce609e10834 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:21.591131 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a71a8-75e9-49cc-aa26-66c6687b2770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:21.591728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a71a8-75e9-49cc-aa26-66c6687b2770 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:21.592009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a71a8-75e9-49cc-aa26-66c6687b2770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:21.592281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a71a8-75e9-49cc-aa26-66c6687b2770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:21.595125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a71a8-75e9-49cc-aa26-66c6687b2770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:21.595125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a71a8-75e9-49cc-aa26-66c6687b2770 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:21.605833 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1132/2293] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:54:21.609951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e84080-e29e-4cc9-ae46-888c8f0ed622 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:21.620047 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e84080-e29e-4cc9-ae46-888c8f0ed622 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:21.624669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93412f-a0eb-42fd-9ed2-270e82706dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:21.625323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93412f-a0eb-42fd-9ed2-270e82706dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:21.625643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93412f-a0eb-42fd-9ed2-270e82706dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:21.625919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93412f-a0eb-42fd-9ed2-270e82706dc1 admin admin] PATH_INFO: `/v3/projects/722d3fff2ee347e3ac44afa7f68ef415/users/64742dfc573b4fd88366e804f632c2c9/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:21.640988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93412f-a0eb-42fd-9ed2-270e82706dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=722d3fff2ee347e3ac44afa7f68ef415, user_id=64742dfc573b4fd88366e804f632c2c9, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:21.643709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93412f-a0eb-42fd-9ed2-270e82706dc1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:21.708579 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1162/2294] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:54:21 2022] PUT /identity/v3/projects/722d3fff2ee347e3ac44afa7f68ef415/users/64742dfc573b4fd88366e804f632c2c9/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:21.720535 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439cc70-8fe3-4029-8a96-1e04c2d1df6b None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:21.720851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439cc70-8fe3-4029-8a96-1e04c2d1df6b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:21.721173 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439cc70-8fe3-4029-8a96-1e04c2d1df6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:21.750324 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1439cc70-8fe3-4029-8a96-1e04c2d1df6b None None] MFA Rules not processed for user `64742dfc573b4fd88366e804f632c2c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:21.751640 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1439cc70-8fe3-4029-8a96-1e04c2d1df6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:21.820785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1133/2295] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:21 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:21.825339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93412f-a0eb-42fd-9ed2-270e82706dc1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:21.897510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93412f-a0eb-42fd-9ed2-270e82706dc1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:21.902943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be6f68-4aa1-4cfc-81d3-b8377a0b305f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:21.903575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be6f68-4aa1-4cfc-81d3-b8377a0b305f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:21.903877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be6f68-4aa1-4cfc-81d3-b8377a0b305f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:21.904133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be6f68-4aa1-4cfc-81d3-b8377a0b305f admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:21.904769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be6f68-4aa1-4cfc-81d3-b8377a0b305f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:21.906715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be6f68-4aa1-4cfc-81d3-b8377a0b305f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:22.002875 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1163/2296] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:54:21 2022] POST /identity/v3/projects => generated 403 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:54:22.007418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439cc70-8fe3-4029-8a96-1e04c2d1df6b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:22.019035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439cc70-8fe3-4029-8a96-1e04c2d1df6b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:22.075286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7517a8c-ea15-4d02-876c-3eb1f3e07f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:22.075810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7517a8c-ea15-4d02-876c-3eb1f3e07f06 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:22.076095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7517a8c-ea15-4d02-876c-3eb1f3e07f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:22.076366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7517a8c-ea15-4d02-876c-3eb1f3e07f06 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:22.076988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7517a8c-ea15-4d02-876c-3eb1f3e07f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:22.078930 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7517a8c-ea15-4d02-876c-3eb1f3e07f06 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:22.172274 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1134/2297] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:54:22 2022] POST /identity/v3/users => generated 308 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:54:22.176303 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be6f68-4aa1-4cfc-81d3-b8377a0b305f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:22.187190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be6f68-4aa1-4cfc-81d3-b8377a0b305f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:22.191927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7adf9-5dd5-4f45-89a2-7ff0c9f07ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:22.192565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7adf9-5dd5-4f45-89a2-7ff0c9f07ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:22.192848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7adf9-5dd5-4f45-89a2-7ff0c9f07ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:22.193121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7adf9-5dd5-4f45-89a2-7ff0c9f07ac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:22.193849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7adf9-5dd5-4f45-89a2-7ff0c9f07ac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:22.196959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7adf9-5dd5-4f45-89a2-7ff0c9f07ac7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:22.207301 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1164/2298] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:54:22.211473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7517a8c-ea15-4d02-876c-3eb1f3e07f06 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:22.222295 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7517a8c-ea15-4d02-876c-3eb1f3e07f06 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:22.226959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3384a0-0f09-4975-a81e-0bf6e6ff724b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:22.227640 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3384a0-0f09-4975-a81e-0bf6e6ff724b admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:22.227922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3384a0-0f09-4975-a81e-0bf6e6ff724b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:22.228197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3384a0-0f09-4975-a81e-0bf6e6ff724b admin admin] PATH_INFO: `/v3/projects/ccf67c76e7bb4de78bef507040e3112d/users/138965c4ea534a4a9314d15563d78156/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:22.243008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3384a0-0f09-4975-a81e-0bf6e6ff724b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf67c76e7bb4de78bef507040e3112d, user_id=138965c4ea534a4a9314d15563d78156, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:22.245801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3384a0-0f09-4975-a81e-0bf6e6ff724b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:22.321554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1135/2299] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:54:22 2022] PUT /identity/v3/projects/ccf67c76e7bb4de78bef507040e3112d/users/138965c4ea534a4a9314d15563d78156/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:22.326524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7adf9-5dd5-4f45-89a2-7ff0c9f07ac7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:22.337049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7adf9-5dd5-4f45-89a2-7ff0c9f07ac7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:22.401076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927f8a8-00e4-4baa-b495-44483089847e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:22.401716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927f8a8-00e4-4baa-b495-44483089847e admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:22.402000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927f8a8-00e4-4baa-b495-44483089847e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:22.402265 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927f8a8-00e4-4baa-b495-44483089847e admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:22.402918 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927f8a8-00e4-4baa-b495-44483089847e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:22.404500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927f8a8-00e4-4baa-b495-44483089847e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:22.414593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1165/2300] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:54:22.419127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3384a0-0f09-4975-a81e-0bf6e6ff724b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:22.431508 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3384a0-0f09-4975-a81e-0bf6e6ff724b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:22.436850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b48c7d-7038-474c-a38d-2a2bc3bbd0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:22.437505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b48c7d-7038-474c-a38d-2a2bc3bbd0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:22.437821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b48c7d-7038-474c-a38d-2a2bc3bbd0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:22.438095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b48c7d-7038-474c-a38d-2a2bc3bbd0c4 admin admin] PATH_INFO: `/v3/projects/ccf67c76e7bb4de78bef507040e3112d/users/138965c4ea534a4a9314d15563d78156/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:22.440221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b48c7d-7038-474c-a38d-2a2bc3bbd0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf67c76e7bb4de78bef507040e3112d, user_id=138965c4ea534a4a9314d15563d78156, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:22.442904 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b48c7d-7038-474c-a38d-2a2bc3bbd0c4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:22.454212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1136/2301] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:54:22 2022] PUT /identity/v3/projects/ccf67c76e7bb4de78bef507040e3112d/users/138965c4ea534a4a9314d15563d78156/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:22.464451 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b03b3-a298-42a6-8a18-f49c3b9664c6 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:22.464741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b03b3-a298-42a6-8a18-f49c3b9664c6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:22.465019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b03b3-a298-42a6-8a18-f49c3b9664c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:22.493740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-500b03b3-a298-42a6-8a18-f49c3b9664c6 None None] MFA Rules not processed for user `138965c4ea534a4a9314d15563d78156`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:22.494949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-500b03b3-a298-42a6-8a18-f49c3b9664c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:22.500397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:22.563305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1166/2302] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:22 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:22.574593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b48c7d-7038-474c-a38d-2a2bc3bbd0c4 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:22.576270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-86b48c7d-7038-474c-a38d-2a2bc3bbd0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:22.586477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:22.633980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b48c7d-7038-474c-a38d-2a2bc3bbd0c4 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:22.638232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75033e4-609e-4ca2-b0dd-a2118b58229a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:22.638828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75033e4-609e-4ca2-b0dd-a2118b58229a service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:22.639105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75033e4-609e-4ca2-b0dd-a2118b58229a service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:22.639377 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75033e4-609e-4ca2-b0dd-a2118b58229a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:22.650826 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75033e4-609e-4ca2-b0dd-a2118b58229a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:22.656156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75033e4-609e-4ca2-b0dd-a2118b58229a service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:22.666631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1137/2303] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:22 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:22.804605 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b03b3-a298-42a6-8a18-f49c3b9664c6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:22.866566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b03b3-a298-42a6-8a18-f49c3b9664c6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:22.871763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:22.872376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:22.872652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:22.872919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] PATH_INFO: `/v3/users/64742dfc573b4fd88366e804f632c2c9` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:22.874048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] RBAC: Authorizing `identity:delete_user(user_id=64742dfc573b4fd88366e804f632c2c9)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:22.875970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:23.084377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64742dfc573b4fd88366e804f632c2c9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:23.178334 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64742dfc573b4fd88366e804f632c2c9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:23.178935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64742dfc573b4fd88366e804f632c2c9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:23.187014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64742dfc573b4fd88366e804f632c2c9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:23.198933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64742dfc573b4fd88366e804f632c2c9'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:23.204975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1167/2304] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:54:22 2022] DELETE /identity/v3/users/64742dfc573b4fd88366e804f632c2c9 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:23.209449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a4dc7-44df-49ff-a2bb-d77630503a55 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:23.210003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a4dc7-44df-49ff-a2bb-d77630503a55 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:23.210003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a4dc7-44df-49ff-a2bb-d77630503a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:23.229142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-753a4dc7-44df-49ff-a2bb-d77630503a55 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:23.231027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-753a4dc7-44df-49ff-a2bb-d77630503a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:23.289665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1138/2305] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:23.300141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:23.301544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:23.328365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:23.359607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f4091-9a4d-4e8d-a612-8e53b48a251d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:23.363350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ce91f-ee5e-40dd-bd2f-56465370ce97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:23.364078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ce91f-ee5e-40dd-bd2f-56465370ce97 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:23.364361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ce91f-ee5e-40dd-bd2f-56465370ce97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:23.364632 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ce91f-ee5e-40dd-bd2f-56465370ce97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:23.373560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ce91f-ee5e-40dd-bd2f-56465370ce97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:23.375047 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ce91f-ee5e-40dd-bd2f-56465370ce97 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:23.384701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1168/2306] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:23.824378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a4dc7-44df-49ff-a2bb-d77630503a55 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:23.825843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-753a4dc7-44df-49ff-a2bb-d77630503a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:23.838177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a4dc7-44df-49ff-a2bb-d77630503a55 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:23.843229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:23.844648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:23.844648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:23.844648 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] PATH_INFO: `/v3/projects/722d3fff2ee347e3ac44afa7f68ef415` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:23.846105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=722d3fff2ee347e3ac44afa7f68ef415)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:23.848188 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:23.973426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '722d3fff2ee347e3ac44afa7f68ef415', 'request_id': 'req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:23.979905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '722d3fff2ee347e3ac44afa7f68ef415', 'request_id': 'req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:23.986170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '722d3fff2ee347e3ac44afa7f68ef415', 'request_id': 'req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:24.096338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] The token cache is being invalidate because project 722d3fff2ee347e3ac44afa7f68ef415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:54:24.096533 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:24.097167 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:24.098754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1139/2307] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:54:23 2022] DELETE /identity/v3/projects/722d3fff2ee347e3ac44afa7f68ef415 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:24.103365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ce91f-ee5e-40dd-bd2f-56465370ce97 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:24.115591 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ce91f-ee5e-40dd-bd2f-56465370ce97 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:24.183533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c369675-fb9e-4b27-9a3a-913ae7e20313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:24.184447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c369675-fb9e-4b27-9a3a-913ae7e20313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:24.184912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c369675-fb9e-4b27-9a3a-913ae7e20313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:24.185350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c369675-fb9e-4b27-9a3a-913ae7e20313 admin admin] PATH_INFO: `/v3/users/138965c4ea534a4a9314d15563d78156` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:24.186643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c369675-fb9e-4b27-9a3a-913ae7e20313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=138965c4ea534a4a9314d15563d78156)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:24.188478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c369675-fb9e-4b27-9a3a-913ae7e20313 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:24.513401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5c369675-fb9e-4b27-9a3a-913ae7e20313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138965c4ea534a4a9314d15563d78156'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:24.691090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5c369675-fb9e-4b27-9a3a-913ae7e20313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138965c4ea534a4a9314d15563d78156'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:24.691829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5c369675-fb9e-4b27-9a3a-913ae7e20313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138965c4ea534a4a9314d15563d78156'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:24.700154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5c369675-fb9e-4b27-9a3a-913ae7e20313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138965c4ea534a4a9314d15563d78156'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:24.715597 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5c369675-fb9e-4b27-9a3a-913ae7e20313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138965c4ea534a4a9314d15563d78156'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:24.721626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1169/2308] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:54:24 2022] DELETE /identity/v3/users/138965c4ea534a4a9314d15563d78156 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:25.162540 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:25.166239 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:25.178935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e68ae-0842-44b2-a5c9-3bbb061a37cf admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:25.211251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:25.237831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:25.238463 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:25.238746 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:25.239019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf admin admin] PATH_INFO: `/v3/projects/ccf67c76e7bb4de78bef507040e3112d` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:25.240141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf67c76e7bb4de78bef507040e3112d)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:25.242078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:25.360525 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf67c76e7bb4de78bef507040e3112d', 'request_id': 'req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:25.367146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf67c76e7bb4de78bef507040e3112d', 'request_id': 'req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:25.373300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf67c76e7bb4de78bef507040e3112d', 'request_id': 'req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:25.484796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf admin admin] The token cache is being invalidate because project ccf67c76e7bb4de78bef507040e3112d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:54:25.488507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:25.488507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4c1b8ec8-dcea-4b12-9ac1-4ae97dc565cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:25.488507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1140/2309] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:54:25 2022] DELETE /identity/v3/projects/ccf67c76e7bb4de78bef507040e3112d => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:25.492114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78842f-8b7a-497b-8568-b515ac7c4155 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:25.492576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78842f-8b7a-497b-8568-b515ac7c4155 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:25.492675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78842f-8b7a-497b-8568-b515ac7c4155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:25.512641 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5d78842f-8b7a-497b-8568-b515ac7c4155 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:25.513934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5d78842f-8b7a-497b-8568-b515ac7c4155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:25.552752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:25.573548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1170/2310] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:25.582104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6871a93-db13-4fd2-b270-2b2eaa8e7e29 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:25.582291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6871a93-db13-4fd2-b270-2b2eaa8e7e29 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:25.582570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6871a93-db13-4fd2-b270-2b2eaa8e7e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:25.609247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d6871a93-db13-4fd2-b270-2b2eaa8e7e29 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:25.610506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d6871a93-db13-4fd2-b270-2b2eaa8e7e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:25.618444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1141/2311] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:25.623496 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78842f-8b7a-497b-8568-b515ac7c4155 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:25.632731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78842f-8b7a-497b-8568-b515ac7c4155 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:25.637715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5894a68-3b0f-47ce-8e80-7ed6ea8c7774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:25.638303 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894a68-3b0f-47ce-8e80-7ed6ea8c7774 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:25.638575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894a68-3b0f-47ce-8e80-7ed6ea8c7774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:25.638841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894a68-3b0f-47ce-8e80-7ed6ea8c7774 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:25.639476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5894a68-3b0f-47ce-8e80-7ed6ea8c7774 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:54:25.639768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5894a68-3b0f-47ce-8e80-7ed6ea8c7774 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:25.641373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5894a68-3b0f-47ce-8e80-7ed6ea8c7774 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:25.653478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1171/2312] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:54:25.657743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6871a93-db13-4fd2-b270-2b2eaa8e7e29 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:25.666909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6871a93-db13-4fd2-b270-2b2eaa8e7e29 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:25.672338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b8346-a8ec-4045-b532-90d7adb774f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:25.672928 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b8346-a8ec-4045-b532-90d7adb774f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:25.673210 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b8346-a8ec-4045-b532-90d7adb774f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:25.673489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b8346-a8ec-4045-b532-90d7adb774f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:25.674169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b8346-a8ec-4045-b532-90d7adb774f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:25.676073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b8346-a8ec-4045-b532-90d7adb774f2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:25.789672 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1142/2313] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:54:25 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:54:25.794321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5894a68-3b0f-47ce-8e80-7ed6ea8c7774 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:25.804090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5894a68-3b0f-47ce-8e80-7ed6ea8c7774 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:25.858472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13755c7d-6a74-4abc-b6ea-254a79404904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:25.858973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13755c7d-6a74-4abc-b6ea-254a79404904 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:25.859248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13755c7d-6a74-4abc-b6ea-254a79404904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:25.859566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13755c7d-6a74-4abc-b6ea-254a79404904 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:25.860160 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13755c7d-6a74-4abc-b6ea-254a79404904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:25.862093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13755c7d-6a74-4abc-b6ea-254a79404904 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:25.943933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1172/2314] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:54:25 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:54:25.948179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b8346-a8ec-4045-b532-90d7adb774f2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:25.958512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b8346-a8ec-4045-b532-90d7adb774f2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:25.963234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd7c10-b1eb-4f87-9252-2bc9fb6daacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:25.965233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd7c10-b1eb-4f87-9252-2bc9fb6daacb admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:25.965233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd7c10-b1eb-4f87-9252-2bc9fb6daacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:25.965233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd7c10-b1eb-4f87-9252-2bc9fb6daacb admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:25.965233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd7c10-b1eb-4f87-9252-2bc9fb6daacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:25.966570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd7c10-b1eb-4f87-9252-2bc9fb6daacb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:25.976371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1143/2315] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:54:25.980845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13755c7d-6a74-4abc-b6ea-254a79404904 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:25.991953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13755c7d-6a74-4abc-b6ea-254a79404904 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:25.999216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319613f7-0895-4352-b526-604e2ebddd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:26.000170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319613f7-0895-4352-b526-604e2ebddd47 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:26.000645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319613f7-0895-4352-b526-604e2ebddd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:26.001000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319613f7-0895-4352-b526-604e2ebddd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:26.001696 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319613f7-0895-4352-b526-604e2ebddd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:26.003345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319613f7-0895-4352-b526-604e2ebddd47 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:26.014064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1173/2316] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:54:26.018273 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd7c10-b1eb-4f87-9252-2bc9fb6daacb admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:26.029304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd7c10-b1eb-4f87-9252-2bc9fb6daacb admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:26.034007 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e989525-5ae3-4057-8281-ad60cf3d8564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:26.034664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e989525-5ae3-4057-8281-ad60cf3d8564 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:26.034949 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e989525-5ae3-4057-8281-ad60cf3d8564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:26.035229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e989525-5ae3-4057-8281-ad60cf3d8564 admin admin] PATH_INFO: `/v3/projects/cee63b98c4f244fbad6aa29ec6a2ea5a/users/f9104c55556a467984e53bb98eeb2421/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:26.050604 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e989525-5ae3-4057-8281-ad60cf3d8564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee63b98c4f244fbad6aa29ec6a2ea5a, user_id=f9104c55556a467984e53bb98eeb2421, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:26.053294 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e989525-5ae3-4057-8281-ad60cf3d8564 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:26.121096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1144/2317] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:54:26 2022] PUT /identity/v3/projects/cee63b98c4f244fbad6aa29ec6a2ea5a/users/f9104c55556a467984e53bb98eeb2421/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:26.126106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335d21e-78d0-4caf-a82a-9d91e44a59a0 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:26.131312 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335d21e-78d0-4caf-a82a-9d91e44a59a0 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:26.131312 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335d21e-78d0-4caf-a82a-9d91e44a59a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:26.145160 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d335d21e-78d0-4caf-a82a-9d91e44a59a0 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:26.146514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d335d21e-78d0-4caf-a82a-9d91e44a59a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:26.204224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1174/2318] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:26.213959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e989525-5ae3-4057-8281-ad60cf3d8564 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:26.216223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7e989525-5ae3-4057-8281-ad60cf3d8564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:26.272892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e989525-5ae3-4057-8281-ad60cf3d8564 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:26.276545 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a490be-2f7d-459f-8742-62f20c95ba58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:26.277152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a490be-2f7d-459f-8742-62f20c95ba58 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:26.278029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a490be-2f7d-459f-8742-62f20c95ba58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:26.278331 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a490be-2f7d-459f-8742-62f20c95ba58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:26.287408 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a490be-2f7d-459f-8742-62f20c95ba58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:26.288902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a490be-2f7d-459f-8742-62f20c95ba58 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:26.299069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1145/2319] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:27.763073 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1c3fc-b0a3-4542-87e1-fa2ce3137776 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:27.763539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1c3fc-b0a3-4542-87e1-fa2ce3137776 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:27.763539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1c3fc-b0a3-4542-87e1-fa2ce3137776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:27.811507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-86b1c3fc-b0a3-4542-87e1-fa2ce3137776 None None] MFA Rules not processed for user `f9104c55556a467984e53bb98eeb2421`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:27.812733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-86b1c3fc-b0a3-4542-87e1-fa2ce3137776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:27.865067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:27.897317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1175/2320] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:54:27.901895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a490be-2f7d-459f-8742-62f20c95ba58 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:27.907482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:27.913442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a490be-2f7d-459f-8742-62f20c95ba58 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:27.918610 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0744f-0374-45c6-a535-ab1c6b5af4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:27.922856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0744f-0374-45c6-a535-ab1c6b5af4fe admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:27.922856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0744f-0374-45c6-a535-ab1c6b5af4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:27.922856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0744f-0374-45c6-a535-ab1c6b5af4fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:27.922856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0744f-0374-45c6-a535-ab1c6b5af4fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:27.922856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0744f-0374-45c6-a535-ab1c6b5af4fe admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:28.042810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1146/2321] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:54:27 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:54:28.050284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1c3fc-b0a3-4542-87e1-fa2ce3137776 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:28.065179 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1c3fc-b0a3-4542-87e1-fa2ce3137776 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:28.125743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f91dd-1116-4362-a028-7ed0b2f3e431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:28.126224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f91dd-1116-4362-a028-7ed0b2f3e431 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:28.126500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f91dd-1116-4362-a028-7ed0b2f3e431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:28.126770 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f91dd-1116-4362-a028-7ed0b2f3e431 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:28.127397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f91dd-1116-4362-a028-7ed0b2f3e431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:28.129344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f91dd-1116-4362-a028-7ed0b2f3e431 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:28.209354 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f70a74-70c4-4d64-960f-27d413a9dbf4 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:28.209682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f70a74-70c4-4d64-960f-27d413a9dbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:28.209985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f70a74-70c4-4d64-960f-27d413a9dbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:28.212915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1176/2322] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:54:28 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:54:28.216885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f91dd-1116-4362-a028-7ed0b2f3e431 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:28.227285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f91dd-1116-4362-a028-7ed0b2f3e431 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:28.231922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f092d-812b-414e-b6cd-cfc44da0b6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:28.232086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c8f70a74-70c4-4d64-960f-27d413a9dbf4 None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:28.232581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f092d-812b-414e-b6cd-cfc44da0b6eb admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:28.232879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f092d-812b-414e-b6cd-cfc44da0b6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:28.233153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f092d-812b-414e-b6cd-cfc44da0b6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:28.233703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c8f70a74-70c4-4d64-960f-27d413a9dbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:28.233837 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f092d-812b-414e-b6cd-cfc44da0b6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:28.235432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f092d-812b-414e-b6cd-cfc44da0b6eb admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:28.255890 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1177/2323] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:54:28.257999 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f092d-812b-414e-b6cd-cfc44da0b6eb admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:28.271710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f092d-812b-414e-b6cd-cfc44da0b6eb admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:28.279342 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcfeb7-c90e-48f9-b033-71dd3ea409f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:28.280321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcfeb7-c90e-48f9-b033-71dd3ea409f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:28.280660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcfeb7-c90e-48f9-b033-71dd3ea409f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:28.281042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcfeb7-c90e-48f9-b033-71dd3ea409f0 admin admin] PATH_INFO: `/v3/projects/86135ffba380490ab1f883ff868d49bf/users/0afeef5b64e7469e9c903cab23d969ed/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:28.297917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcfeb7-c90e-48f9-b033-71dd3ea409f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86135ffba380490ab1f883ff868d49bf, user_id=0afeef5b64e7469e9c903cab23d969ed, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:28.303242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcfeb7-c90e-48f9-b033-71dd3ea409f0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:28.305493 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1147/2324] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:54:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:54:28.406432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1178/2325] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:54:28 2022] PUT /identity/v3/projects/86135ffba380490ab1f883ff868d49bf/users/0afeef5b64e7469e9c903cab23d969ed/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:54:28.410552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f70a74-70c4-4d64-960f-27d413a9dbf4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:28.422397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f70a74-70c4-4d64-960f-27d413a9dbf4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:28.479165 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da09f5-dfcc-44bb-b223-3fd933e266f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:28.479840 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da09f5-dfcc-44bb-b223-3fd933e266f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:28.480120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da09f5-dfcc-44bb-b223-3fd933e266f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:28.480903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da09f5-dfcc-44bb-b223-3fd933e266f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:28.481572 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da09f5-dfcc-44bb-b223-3fd933e266f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:28.483169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da09f5-dfcc-44bb-b223-3fd933e266f1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:28.494249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1148/2326] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:54:28.499556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcfeb7-c90e-48f9-b033-71dd3ea409f0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:28.503069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da09f5-dfcc-44bb-b223-3fd933e266f1 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:28.505154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c5da09f5-dfcc-44bb-b223-3fd933e266f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:28.511987 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcfeb7-c90e-48f9-b033-71dd3ea409f0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:28.517003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783a625-0aad-4c7a-aa2d-4ebc1fc32cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:28.517735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783a625-0aad-4c7a-aa2d-4ebc1fc32cdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:28.518016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783a625-0aad-4c7a-aa2d-4ebc1fc32cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:28.518294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783a625-0aad-4c7a-aa2d-4ebc1fc32cdf admin admin] PATH_INFO: `/v3/projects/86135ffba380490ab1f883ff868d49bf/users/0afeef5b64e7469e9c903cab23d969ed/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:28.520325 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783a625-0aad-4c7a-aa2d-4ebc1fc32cdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=86135ffba380490ab1f883ff868d49bf, user_id=0afeef5b64e7469e9c903cab23d969ed, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:28.523027 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783a625-0aad-4c7a-aa2d-4ebc1fc32cdf admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:28.538008 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1179/2327] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:54:28 2022] PUT /identity/v3/projects/86135ffba380490ab1f883ff868d49bf/users/0afeef5b64e7469e9c903cab23d969ed/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:28.588347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da09f5-dfcc-44bb-b223-3fd933e266f1 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:28.653449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76116e0-df17-4094-99cb-2e25994415ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:28.654637 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76116e0-df17-4094-99cb-2e25994415ce service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:28.655018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76116e0-df17-4094-99cb-2e25994415ce service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:28.655389 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76116e0-df17-4094-99cb-2e25994415ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:28.725802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76116e0-df17-4094-99cb-2e25994415ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:28.727440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76116e0-df17-4094-99cb-2e25994415ce service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:28.737015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1149/2328] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:54:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 18:54:28.765466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:29.577337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783a625-0aad-4c7a-aa2d-4ebc1fc32cdf admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:29.579062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0783a625-0aad-4c7a-aa2d-4ebc1fc32cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:29.679081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783a625-0aad-4c7a-aa2d-4ebc1fc32cdf admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:29.685130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f0178-0c34-4861-9f9c-7b5dfa6740f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:29.685858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f0178-0c34-4861-9f9c-7b5dfa6740f6 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:29.691168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f0178-0c34-4861-9f9c-7b5dfa6740f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:29.691168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f0178-0c34-4861-9f9c-7b5dfa6740f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:29.691168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7c9f0178-0c34-4861-9f9c-7b5dfa6740f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:29.711082 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:29.804687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f0178-0c34-4861-9f9c-7b5dfa6740f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:29.808975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f0178-0c34-4861-9f9c-7b5dfa6740f6 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:29.831684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1180/2329] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:30.145339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f59e7-9742-41c3-9f2d-380df94164e5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:30.145901 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f59e7-9742-41c3-9f2d-380df94164e5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:30.145901 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f59e7-9742-41c3-9f2d-380df94164e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:30.203925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ca0f59e7-9742-41c3-9f2d-380df94164e5 None None] MFA Rules not processed for user `0afeef5b64e7469e9c903cab23d969ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:30.205235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ca0f59e7-9742-41c3-9f2d-380df94164e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:30.352970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1150/2330] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:30.509856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f0178-0c34-4861-9f9c-7b5dfa6740f6 service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:30.515339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7c9f0178-0c34-4861-9f9c-7b5dfa6740f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:30.582943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f0178-0c34-4861-9f9c-7b5dfa6740f6 service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:30.586898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cc6ec-9113-4cbb-8906-61455948613c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:30.587568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cc6ec-9113-4cbb-8906-61455948613c service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:30.587909 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cc6ec-9113-4cbb-8906-61455948613c service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:30.588180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cc6ec-9113-4cbb-8906-61455948613c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:30.662524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2cc6ec-9113-4cbb-8906-61455948613c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:30.664147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2cc6ec-9113-4cbb-8906-61455948613c service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:30.675287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1181/2331] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:32.543013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f59e7-9742-41c3-9f2d-380df94164e5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:32.543013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ca0f59e7-9742-41c3-9f2d-380df94164e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:32.557456 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f59e7-9742-41c3-9f2d-380df94164e5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:32.562342 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821674f-96b9-4f2e-bf9d-c3397b102f85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:32.563073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821674f-96b9-4f2e-bf9d-c3397b102f85 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:32.563352 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821674f-96b9-4f2e-bf9d-c3397b102f85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:32.563786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821674f-96b9-4f2e-bf9d-c3397b102f85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:32.578321 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821674f-96b9-4f2e-bf9d-c3397b102f85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:32.579895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821674f-96b9-4f2e-bf9d-c3397b102f85 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:32.590827 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1151/2332] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:40.670273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=67 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:42.585868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=83 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:44.732770 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cc6ec-9113-4cbb-8906-61455948613c service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:44.736369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7f2cc6ec-9113-4cbb-8906-61455948613c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:44.753636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cc6ec-9113-4cbb-8906-61455948613c service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:44.759622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab4cc0-638a-4b5b-8cd3-54589fadc16b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:44.760333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab4cc0-638a-4b5b-8cd3-54589fadc16b service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:44.761736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab4cc0-638a-4b5b-8cd3-54589fadc16b service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:44.761975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab4cc0-638a-4b5b-8cd3-54589fadc16b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:44.763808 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-02ab4cc0-638a-4b5b-8cd3-54589fadc16b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:44.776038 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab4cc0-638a-4b5b-8cd3-54589fadc16b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:44.777757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab4cc0-638a-4b5b-8cd3-54589fadc16b service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:44.789191 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1182/2333] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:46.139637 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821674f-96b9-4f2e-bf9d-c3397b102f85 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:46.141180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-0821674f-96b9-4f2e-bf9d-c3397b102f85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:46.198186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821674f-96b9-4f2e-bf9d-c3397b102f85 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:46.201889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b8ce7-f422-4cad-8628-bb0560ed4cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:46.202482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b8ce7-f422-4cad-8628-bb0560ed4cda service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:46.202762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b8ce7-f422-4cad-8628-bb0560ed4cda service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:46.203029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b8ce7-f422-4cad-8628-bb0560ed4cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:46.205952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ea0b8ce7-f422-4cad-8628-bb0560ed4cda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:46.216046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b8ce7-f422-4cad-8628-bb0560ed4cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:46.217655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b8ce7-f422-4cad-8628-bb0560ed4cda service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:46.226125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1152/2334] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:47.172586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab4cc0-638a-4b5b-8cd3-54589fadc16b service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:47.174077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-02ab4cc0-638a-4b5b-8cd3-54589fadc16b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:47.242322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab4cc0-638a-4b5b-8cd3-54589fadc16b service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:47.247325 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a6b68-7571-44f8-bf9d-51d540a15f22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0aa6b215f646c280af245a89ccf849', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:47.247924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a6b68-7571-44f8-bf9d-51d540a15f22 service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:47.248203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a6b68-7571-44f8-bf9d-51d540a15f22 service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:47.248472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a6b68-7571-44f8-bf9d-51d540a15f22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:47.250709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-055a6b68-7571-44f8-bf9d-51d540a15f22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:47.262045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a6b68-7571-44f8-bf9d-51d540a15f22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:47.263567 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a6b68-7571-44f8-bf9d-51d540a15f22 service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:47.276166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1183/2335] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:49.694112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b8ce7-f422-4cad-8628-bb0560ed4cda service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:49.814532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b8ce7-f422-4cad-8628-bb0560ed4cda service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:49.821341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196a6ac-4ab2-4d2a-9837-4f997963834f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:49.822402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196a6ac-4ab2-4d2a-9837-4f997963834f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:49.823167 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196a6ac-4ab2-4d2a-9837-4f997963834f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:49.823654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196a6ac-4ab2-4d2a-9837-4f997963834f admin admin] PATH_INFO: `/v3/users/f9104c55556a467984e53bb98eeb2421` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:49.826261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196a6ac-4ab2-4d2a-9837-4f997963834f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9104c55556a467984e53bb98eeb2421)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:49.829572 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196a6ac-4ab2-4d2a-9837-4f997963834f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:49.852953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a6b68-7571-44f8-bf9d-51d540a15f22 service placement] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:49.869598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a6b68-7571-44f8-bf9d-51d540a15f22 service placement] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:49.882172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc981a-4073-4d9e-ad50-abec58d83b4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:49.882172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc981a-4073-4d9e-ad50-abec58d83b4a service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:49.882172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc981a-4073-4d9e-ad50-abec58d83b4a service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:49.882172 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc981a-4073-4d9e-ad50-abec58d83b4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:49.882563 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0cbc981a-4073-4d9e-ad50-abec58d83b4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:49.896066 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc981a-4073-4d9e-ad50-abec58d83b4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:49.897908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc981a-4073-4d9e-ad50-abec58d83b4a service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:49.913742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1184/2336] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:54:50.118046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4196a6ac-4ab2-4d2a-9837-4f997963834f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9104c55556a467984e53bb98eeb2421'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:50.207326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4196a6ac-4ab2-4d2a-9837-4f997963834f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9104c55556a467984e53bb98eeb2421'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:50.207992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4196a6ac-4ab2-4d2a-9837-4f997963834f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9104c55556a467984e53bb98eeb2421'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:50.219341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4196a6ac-4ab2-4d2a-9837-4f997963834f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9104c55556a467984e53bb98eeb2421'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:50.237004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4196a6ac-4ab2-4d2a-9837-4f997963834f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9104c55556a467984e53bb98eeb2421'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:50.251808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1153/2337] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:54:49 2022] DELETE /identity/v3/users/f9104c55556a467984e53bb98eeb2421 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:50.576495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc981a-4073-4d9e-ad50-abec58d83b4a service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:50.611889 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc981a-4073-4d9e-ad50-abec58d83b4a service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:50.737568 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:50.738324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:50.738673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:50.739015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] PATH_INFO: `/v3/projects/cee63b98c4f244fbad6aa29ec6a2ea5a` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:50.740266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee63b98c4f244fbad6aa29ec6a2ea5a)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:50.742352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:50.860063 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730464c3-b61a-44aa-920c-ae5ff5e89678 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:50.860613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730464c3-b61a-44aa-920c-ae5ff5e89678 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:50.860769 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730464c3-b61a-44aa-920c-ae5ff5e89678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:50.880940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee63b98c4f244fbad6aa29ec6a2ea5a', 'request_id': 'req-69f34e3e-5c66-483c-8e8d-253a2ad3a148'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:50.887965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee63b98c4f244fbad6aa29ec6a2ea5a', 'request_id': 'req-69f34e3e-5c66-483c-8e8d-253a2ad3a148'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:50.893076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-730464c3-b61a-44aa-920c-ae5ff5e89678 None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:50.893076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-730464c3-b61a-44aa-920c-ae5ff5e89678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:50.900096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee63b98c4f244fbad6aa29ec6a2ea5a', 'request_id': 'req-69f34e3e-5c66-483c-8e8d-253a2ad3a148'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:50.905588 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:50.912594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:50.956159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1154/2338] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:54:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:54:50.969828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730464c3-b61a-44aa-920c-ae5ff5e89678 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:50.971915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-730464c3-b61a-44aa-920c-ae5ff5e89678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:51.022269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] The token cache is being invalidate because project cee63b98c4f244fbad6aa29ec6a2ea5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:54:51.023220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:51.024826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:51.027234 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1185/2339] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:54:50 2022] DELETE /identity/v3/projects/cee63b98c4f244fbad6aa29ec6a2ea5a => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:51.033566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:51.036561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:51.083846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730464c3-b61a-44aa-920c-ae5ff5e89678 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:51.083846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8190f65-f283-454c-b391-0fbda4112f53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:51.083846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8190f65-f283-454c-b391-0fbda4112f53 service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:51.083846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8190f65-f283-454c-b391-0fbda4112f53 service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:51.084417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8190f65-f283-454c-b391-0fbda4112f53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:51.085228 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e8190f65-f283-454c-b391-0fbda4112f53 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:51.097252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8190f65-f283-454c-b391-0fbda4112f53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:51.098926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8190f65-f283-454c-b391-0fbda4112f53 service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:51.107639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1155/2340] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:54:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 18:54:51.134943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f34e3e-5c66-483c-8e8d-253a2ad3a148 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:51.139817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13abc9e-5dfb-4859-9c90-99e8f8186130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:51.140449 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13abc9e-5dfb-4859-9c90-99e8f8186130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:51.140726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13abc9e-5dfb-4859-9c90-99e8f8186130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:51.140993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13abc9e-5dfb-4859-9c90-99e8f8186130 admin admin] PATH_INFO: `/v3/users/0afeef5b64e7469e9c903cab23d969ed` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:51.142145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13abc9e-5dfb-4859-9c90-99e8f8186130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0afeef5b64e7469e9c903cab23d969ed)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:51.144174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13abc9e-5dfb-4859-9c90-99e8f8186130 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:51.371022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d13abc9e-5dfb-4859-9c90-99e8f8186130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0afeef5b64e7469e9c903cab23d969ed'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:51.441492 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d13abc9e-5dfb-4859-9c90-99e8f8186130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0afeef5b64e7469e9c903cab23d969ed'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:51.443029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d13abc9e-5dfb-4859-9c90-99e8f8186130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0afeef5b64e7469e9c903cab23d969ed'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:51.457339 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d13abc9e-5dfb-4859-9c90-99e8f8186130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0afeef5b64e7469e9c903cab23d969ed'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:51.472282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d13abc9e-5dfb-4859-9c90-99e8f8186130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0afeef5b64e7469e9c903cab23d969ed'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:51.479286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1186/2341] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:54:51 2022] DELETE /identity/v3/users/0afeef5b64e7469e9c903cab23d969ed => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:51.852847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8190f65-f283-454c-b391-0fbda4112f53 service swift] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:51.864804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8190f65-f283-454c-b391-0fbda4112f53 service swift] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:51.957388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f123d9-a378-450d-a5ec-923fee8c810b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:51.958107 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f123d9-a378-450d-a5ec-923fee8c810b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:51.958454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f123d9-a378-450d-a5ec-923fee8c810b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:51.958730 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f123d9-a378-450d-a5ec-923fee8c810b admin admin] PATH_INFO: `/v3/projects/86135ffba380490ab1f883ff868d49bf` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:51.959890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f123d9-a378-450d-a5ec-923fee8c810b admin admin] RBAC: Authorizing `identity:delete_project(project_id=86135ffba380490ab1f883ff868d49bf)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:51.962570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f123d9-a378-450d-a5ec-923fee8c810b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:52.113454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-14f123d9-a378-450d-a5ec-923fee8c810b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86135ffba380490ab1f883ff868d49bf', 'request_id': 'req-14f123d9-a378-450d-a5ec-923fee8c810b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:52.130782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-14f123d9-a378-450d-a5ec-923fee8c810b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86135ffba380490ab1f883ff868d49bf', 'request_id': 'req-14f123d9-a378-450d-a5ec-923fee8c810b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:52.145426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:52.149261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-14f123d9-a378-450d-a5ec-923fee8c810b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86135ffba380490ab1f883ff868d49bf', 'request_id': 'req-14f123d9-a378-450d-a5ec-923fee8c810b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:52.246500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-14f123d9-a378-450d-a5ec-923fee8c810b admin admin] The token cache is being invalidate because project 86135ffba380490ab1f883ff868d49bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:54:52.248469 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-14f123d9-a378-450d-a5ec-923fee8c810b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:52.250986 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-14f123d9-a378-450d-a5ec-923fee8c810b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:54:52.255086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1156/2342] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:54:51 2022] DELETE /identity/v3/projects/86135ffba380490ab1f883ff868d49bf => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:54:52.260147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763af58-74dc-4a87-a10c-e3b94ebb63ff None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:52.261478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763af58-74dc-4a87-a10c-e3b94ebb63ff None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:52.261935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763af58-74dc-4a87-a10c-e3b94ebb63ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:52.302696 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-7763af58-74dc-4a87-a10c-e3b94ebb63ff None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:52.304898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7763af58-74dc-4a87-a10c-e3b94ebb63ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:52.419029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1187/2343] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:52.427111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0b555-acbc-48c2-9898-bc89951977ef None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:52.427457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0b555-acbc-48c2-9898-bc89951977ef None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:52.427843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0b555-acbc-48c2-9898-bc89951977ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:52.466783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-67f0b555-acbc-48c2-9898-bc89951977ef None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:52.467955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-67f0b555-acbc-48c2-9898-bc89951977ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:52.477383 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1157/2344] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:52.482029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763af58-74dc-4a87-a10c-e3b94ebb63ff None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:52.496698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763af58-74dc-4a87-a10c-e3b94ebb63ff None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:52.504273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bc1ed-0a9a-4daf-9f9f-dcfbd542de76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:52.505946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bc1ed-0a9a-4daf-9f9f-dcfbd542de76 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:52.506064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bc1ed-0a9a-4daf-9f9f-dcfbd542de76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:52.506498 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bc1ed-0a9a-4daf-9f9f-dcfbd542de76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:52.507556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bc1ed-0a9a-4daf-9f9f-dcfbd542de76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:54:52.508019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bc1ed-0a9a-4daf-9f9f-dcfbd542de76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:52.510775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bc1ed-0a9a-4daf-9f9f-dcfbd542de76 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:52.527554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:54:52.536376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1188/2345] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:54:52.540366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0b555-acbc-48c2-9898-bc89951977ef None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:52.550516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0b555-acbc-48c2-9898-bc89951977ef None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:52.556832 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dca8f-2f9e-42d2-a10c-d2c8b6cd6312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:52.558029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dca8f-2f9e-42d2-a10c-d2c8b6cd6312 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:52.558525 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dca8f-2f9e-42d2-a10c-d2c8b6cd6312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:52.571841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dca8f-2f9e-42d2-a10c-d2c8b6cd6312 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:52.571841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0dca8f-2f9e-42d2-a10c-d2c8b6cd6312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:52.571841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0dca8f-2f9e-42d2-a10c-d2c8b6cd6312 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:52.711919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1158/2346] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:54:52 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:54:52.713264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bc1ed-0a9a-4daf-9f9f-dcfbd542de76 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:52.729237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bc1ed-0a9a-4daf-9f9f-dcfbd542de76 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:52.822269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac9bce-dbf1-4a80-aae1-893a587711cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:52.823336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac9bce-dbf1-4a80-aae1-893a587711cf admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:52.823674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac9bce-dbf1-4a80-aae1-893a587711cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:52.823987 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac9bce-dbf1-4a80-aae1-893a587711cf admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:52.824686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac9bce-dbf1-4a80-aae1-893a587711cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:52.828514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac9bce-dbf1-4a80-aae1-893a587711cf admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:52.956619 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dca8f-2f9e-42d2-a10c-d2c8b6cd6312 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:52.961955 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1189/2347] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:54:52 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:54:52.968471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dca8f-2f9e-42d2-a10c-d2c8b6cd6312 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:52.976755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6a1fb-cd9b-41fb-af9e-2e8983ae8665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:52.977613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6a1fb-cd9b-41fb-af9e-2e8983ae8665 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:52.978158 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6a1fb-cd9b-41fb-af9e-2e8983ae8665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:52.978545 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6a1fb-cd9b-41fb-af9e-2e8983ae8665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:52.979368 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6a1fb-cd9b-41fb-af9e-2e8983ae8665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:52.981156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6a1fb-cd9b-41fb-af9e-2e8983ae8665 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:52.994629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1159/2348] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:54:52.999732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac9bce-dbf1-4a80-aae1-893a587711cf admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:53.011414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac9bce-dbf1-4a80-aae1-893a587711cf admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:53.029000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208dddf-b87d-4769-a59a-bee972c6074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:53.029649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208dddf-b87d-4769-a59a-bee972c6074d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:53.029927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208dddf-b87d-4769-a59a-bee972c6074d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:53.030735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208dddf-b87d-4769-a59a-bee972c6074d admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:53.031424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208dddf-b87d-4769-a59a-bee972c6074d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:53.033261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208dddf-b87d-4769-a59a-bee972c6074d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:53.056643 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1190/2349] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:54:53.056758 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6a1fb-cd9b-41fb-af9e-2e8983ae8665 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:53.077247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6a1fb-cd9b-41fb-af9e-2e8983ae8665 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:53.089557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f9aa7-d5c8-41fc-8bb8-3c30688e0dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:53.090353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f9aa7-d5c8-41fc-8bb8-3c30688e0dbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:53.090643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f9aa7-d5c8-41fc-8bb8-3c30688e0dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:53.090925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f9aa7-d5c8-41fc-8bb8-3c30688e0dbd admin admin] PATH_INFO: `/v3/projects/23abce61b1ad43d3a6ab32fc4fc8eead/users/6073772e5caf49a3acf7ab607452339e/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:53.111796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f9aa7-d5c8-41fc-8bb8-3c30688e0dbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=23abce61b1ad43d3a6ab32fc4fc8eead, user_id=6073772e5caf49a3acf7ab607452339e, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:53.114534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f9aa7-d5c8-41fc-8bb8-3c30688e0dbd admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:53.219874 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1160/2350] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:54:53 2022] PUT /identity/v3/projects/23abce61b1ad43d3a6ab32fc4fc8eead/users/6073772e5caf49a3acf7ab607452339e/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:54:53.222100 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2a6e1-60c9-421a-af59-729c01bb177a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:53.222664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2a6e1-60c9-421a-af59-729c01bb177a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:53.223135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2a6e1-60c9-421a-af59-729c01bb177a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:53.295794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c8b2a6e1-60c9-421a-af59-729c01bb177a None None] MFA Rules not processed for user `6073772e5caf49a3acf7ab607452339e`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:53.297563 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c8b2a6e1-60c9-421a-af59-729c01bb177a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:53.415957 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1191/2351] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:53.430681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f9aa7-d5c8-41fc-8bb8-3c30688e0dbd admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:53.430681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-837f9aa7-d5c8-41fc-8bb8-3c30688e0dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:53.533571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f9aa7-d5c8-41fc-8bb8-3c30688e0dbd admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:53.537471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299c2e6-048a-4441-b214-f453fb9b6951 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:53.538124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c2e6-048a-4441-b214-f453fb9b6951 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:53.538446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c2e6-048a-4441-b214-f453fb9b6951 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:53.538720 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c2e6-048a-4441-b214-f453fb9b6951 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:53.548252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299c2e6-048a-4441-b214-f453fb9b6951 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:53.555750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299c2e6-048a-4441-b214-f453fb9b6951 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:53.561567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1161/2352] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:54:55.847523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d1223-028c-4fdc-9b2b-212cfdff6143 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:55.849945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d1223-028c-4fdc-9b2b-212cfdff6143 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:55.849945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d1223-028c-4fdc-9b2b-212cfdff6143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:55.891746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-245d1223-028c-4fdc-9b2b-212cfdff6143 None None] MFA Rules not processed for user `00491255f4b44125b6dc9f816c9bbb58`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:54:55.899478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-245d1223-028c-4fdc-9b2b-212cfdff6143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:56.017694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1192/2353] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:54:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:54:56.405256 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299c2e6-048a-4441-b214-f453fb9b6951 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:54:56.406784 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4299c2e6-048a-4441-b214-f453fb9b6951 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:56.481828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299c2e6-048a-4441-b214-f453fb9b6951 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:54:56.487192 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ea7a9-c2f2-4b26-919a-53ca36916cb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:54:56.488215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ea7a9-c2f2-4b26-919a-53ca36916cb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:54:56.488782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ea7a9-c2f2-4b26-919a-53ca36916cb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:54:56.489222 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ea7a9-c2f2-4b26-919a-53ca36916cb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:54:56.491712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-568ea7a9-c2f2-4b26-919a-53ca36916cb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:54:56.504474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ea7a9-c2f2-4b26-919a-53ca36916cb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:54:56.507219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ea7a9-c2f2-4b26-919a-53ca36916cb9 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:54:56.517125 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1162/2354] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:54:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:04.236829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d1223-028c-4fdc-9b2b-212cfdff6143 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:04.263327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:04.330074 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d1223-028c-4fdc-9b2b-212cfdff6143 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:04.332831 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:04.333477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:04.333787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:04.334530 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] PATH_INFO: `/v3/users/6073772e5caf49a3acf7ab607452339e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:04.335223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6073772e5caf49a3acf7ab607452339e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:04.337313 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:04.537489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6073772e5caf49a3acf7ab607452339e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:04.611278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6073772e5caf49a3acf7ab607452339e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:04.612006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6073772e5caf49a3acf7ab607452339e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:04.620043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6073772e5caf49a3acf7ab607452339e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:04.632530 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6073772e5caf49a3acf7ab607452339e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:04.639146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1193/2355] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:55:04 2022] DELETE /identity/v3/users/6073772e5caf49a3acf7ab607452339e => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:04.644287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665b408-891e-4c32-992b-87f1d4e1a999 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:04.644835 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665b408-891e-4c32-992b-87f1d4e1a999 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:04.644835 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665b408-891e-4c32-992b-87f1d4e1a999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:04.665229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5665b408-891e-4c32-992b-87f1d4e1a999 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:04.666487 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5665b408-891e-4c32-992b-87f1d4e1a999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:04.694885 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:04.769857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1163/2356] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:04.785591 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:04.788766 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:04.880400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c6146-b192-458e-a5c3-ddbe7895036d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:04.884514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613148c0-2d3d-4a92-a888-7be697a29e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:04.885840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613148c0-2d3d-4a92-a888-7be697a29e37 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:04.886184 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613148c0-2d3d-4a92-a888-7be697a29e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:04.886466 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613148c0-2d3d-4a92-a888-7be697a29e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:04.904386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613148c0-2d3d-4a92-a888-7be697a29e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:04.906168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613148c0-2d3d-4a92-a888-7be697a29e37 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:04.918998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1194/2357] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:06.355866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665b408-891e-4c32-992b-87f1d4e1a999 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:06.358085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5665b408-891e-4c32-992b-87f1d4e1a999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:06.385237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665b408-891e-4c32-992b-87f1d4e1a999 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:06.396903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a121b94-f618-4646-9cbb-e092a3185f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:06.397912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a121b94-f618-4646-9cbb-e092a3185f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:06.398191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a121b94-f618-4646-9cbb-e092a3185f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:06.398455 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a121b94-f618-4646-9cbb-e092a3185f97 admin admin] PATH_INFO: `/v3/projects/23abce61b1ad43d3a6ab32fc4fc8eead` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:06.399992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a121b94-f618-4646-9cbb-e092a3185f97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23abce61b1ad43d3a6ab32fc4fc8eead)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:06.402888 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a121b94-f618-4646-9cbb-e092a3185f97 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:06.566284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6a121b94-f618-4646-9cbb-e092a3185f97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23abce61b1ad43d3a6ab32fc4fc8eead', 'request_id': 'req-6a121b94-f618-4646-9cbb-e092a3185f97'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:06.574166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6a121b94-f618-4646-9cbb-e092a3185f97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23abce61b1ad43d3a6ab32fc4fc8eead', 'request_id': 'req-6a121b94-f618-4646-9cbb-e092a3185f97'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:06.581698 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6a121b94-f618-4646-9cbb-e092a3185f97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23abce61b1ad43d3a6ab32fc4fc8eead', 'request_id': 'req-6a121b94-f618-4646-9cbb-e092a3185f97'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:06.679922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6a121b94-f618-4646-9cbb-e092a3185f97 admin admin] The token cache is being invalidate because project 23abce61b1ad43d3a6ab32fc4fc8eead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:55:06.680267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6a121b94-f618-4646-9cbb-e092a3185f97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:06.680938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6a121b94-f618-4646-9cbb-e092a3185f97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:06.683494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1164/2358] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:55:06 2022] DELETE /identity/v3/projects/23abce61b1ad43d3a6ab32fc4fc8eead => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:06.691109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0efaa-06c0-4e21-9a88-bfa9dd7c705f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:06.692338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0efaa-06c0-4e21-9a88-bfa9dd7c705f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:06.692838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0efaa-06c0-4e21-9a88-bfa9dd7c705f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:06.722378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cfc0efaa-06c0-4e21-9a88-bfa9dd7c705f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:06.724285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cfc0efaa-06c0-4e21-9a88-bfa9dd7c705f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:06.766775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:06.803914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1195/2359] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:06.812940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d674c11-8349-4d3b-8ab7-1e377c870b84 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:06.813417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d674c11-8349-4d3b-8ab7-1e377c870b84 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:06.813651 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d674c11-8349-4d3b-8ab7-1e377c870b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:06.837141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4d674c11-8349-4d3b-8ab7-1e377c870b84 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:06.838418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4d674c11-8349-4d3b-8ab7-1e377c870b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:06.846754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1165/2360] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:06.850993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0efaa-06c0-4e21-9a88-bfa9dd7c705f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:06.862152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0efaa-06c0-4e21-9a88-bfa9dd7c705f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:06.868109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113d044-eded-4859-8012-23e70a6775a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:06.868763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113d044-eded-4859-8012-23e70a6775a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:06.869040 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113d044-eded-4859-8012-23e70a6775a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:06.869309 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113d044-eded-4859-8012-23e70a6775a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:06.870077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113d044-eded-4859-8012-23e70a6775a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:55:06.870429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113d044-eded-4859-8012-23e70a6775a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:06.872170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113d044-eded-4859-8012-23e70a6775a5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:06.885262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1196/2361] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:55:06.889667 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d674c11-8349-4d3b-8ab7-1e377c870b84 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:06.899393 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d674c11-8349-4d3b-8ab7-1e377c870b84 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:06.904184 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838e19e-7afc-41a9-97e6-13f19a0e70c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:06.904777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838e19e-7afc-41a9-97e6-13f19a0e70c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:06.905057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838e19e-7afc-41a9-97e6-13f19a0e70c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:06.905330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838e19e-7afc-41a9-97e6-13f19a0e70c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:06.906044 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838e19e-7afc-41a9-97e6-13f19a0e70c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:06.908059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838e19e-7afc-41a9-97e6-13f19a0e70c3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:06.997492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1166/2362] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:55:06 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:07.001737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113d044-eded-4859-8012-23e70a6775a5 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:07.011937 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113d044-eded-4859-8012-23e70a6775a5 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:07.070823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bb2e8-747b-4a2e-974a-13d84b014956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:07.071423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bb2e8-747b-4a2e-974a-13d84b014956 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:07.071696 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bb2e8-747b-4a2e-974a-13d84b014956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:07.071962 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bb2e8-747b-4a2e-974a-13d84b014956 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:07.072595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bb2e8-747b-4a2e-974a-13d84b014956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:07.074686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bb2e8-747b-4a2e-974a-13d84b014956 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:07.185201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1197/2363] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:55:07 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:07.188970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838e19e-7afc-41a9-97e6-13f19a0e70c3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:07.199646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838e19e-7afc-41a9-97e6-13f19a0e70c3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:07.204602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab11d0-7a94-4ff8-8171-4953f9607932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:07.205619 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab11d0-7a94-4ff8-8171-4953f9607932 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:07.210339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab11d0-7a94-4ff8-8171-4953f9607932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:07.210339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab11d0-7a94-4ff8-8171-4953f9607932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:07.210339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab11d0-7a94-4ff8-8171-4953f9607932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:07.210339 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab11d0-7a94-4ff8-8171-4953f9607932 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:07.220543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1167/2364] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:07.224692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bb2e8-747b-4a2e-974a-13d84b014956 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:07.235322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bb2e8-747b-4a2e-974a-13d84b014956 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:07.239985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3670ca8-e3fb-48b4-839d-ecdb8ec2be4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:07.240525 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3670ca8-e3fb-48b4-839d-ecdb8ec2be4f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:07.240805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3670ca8-e3fb-48b4-839d-ecdb8ec2be4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:07.241076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3670ca8-e3fb-48b4-839d-ecdb8ec2be4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:07.241746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3670ca8-e3fb-48b4-839d-ecdb8ec2be4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:07.243336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3670ca8-e3fb-48b4-839d-ecdb8ec2be4f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:07.253818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1198/2365] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:07.258252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab11d0-7a94-4ff8-8171-4953f9607932 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:07.268523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab11d0-7a94-4ff8-8171-4953f9607932 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:07.273272 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104cc18-059d-4609-890a-61e4dfccf5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:07.274008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104cc18-059d-4609-890a-61e4dfccf5e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:07.274293 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104cc18-059d-4609-890a-61e4dfccf5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:07.274573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104cc18-059d-4609-890a-61e4dfccf5e1 admin admin] PATH_INFO: `/v3/projects/1a178a776de44a638b9cb903a597f789/users/4754b4dd80e244be9e15121d0ebfefe1/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:07.289878 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104cc18-059d-4609-890a-61e4dfccf5e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a178a776de44a638b9cb903a597f789, user_id=4754b4dd80e244be9e15121d0ebfefe1, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:07.292535 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104cc18-059d-4609-890a-61e4dfccf5e1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:07.334316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0c225-e5a7-40e6-bd1d-9a67916e7ece None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:07.334610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0c225-e5a7-40e6-bd1d-9a67916e7ece None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:07.334865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0c225-e5a7-40e6-bd1d-9a67916e7ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:07.355045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c4e0c225-e5a7-40e6-bd1d-9a67916e7ece None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:07.356271 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c4e0c225-e5a7-40e6-bd1d-9a67916e7ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:07.372997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1168/2366] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:07 2022] PUT /identity/v3/projects/1a178a776de44a638b9cb903a597f789/users/4754b4dd80e244be9e15121d0ebfefe1/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:07.384306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fb158-365d-4d5c-8378-ecffec90159a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:07.384684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fb158-365d-4d5c-8378-ecffec90159a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:07.384935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fb158-365d-4d5c-8378-ecffec90159a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:07.419563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-927fb158-365d-4d5c-8378-ecffec90159a None None] MFA Rules not processed for user `4754b4dd80e244be9e15121d0ebfefe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:07.421129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-927fb158-365d-4d5c-8378-ecffec90159a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:07.429962 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1199/2367] 10.222.0.57 () {58 vars in 967 bytes} [Tue Aug 9 18:55:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:55:07.439617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0c225-e5a7-40e6-bd1d-9a67916e7ece None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:07.442575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c4e0c225-e5a7-40e6-bd1d-9a67916e7ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:07.476676 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:07.503205 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:07.509306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1169/2368] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:07.518816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fb158-365d-4d5c-8378-ecffec90159a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:07.520358 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-927fb158-365d-4d5c-8378-ecffec90159a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:07.526461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0c225-e5a7-40e6-bd1d-9a67916e7ece None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:07.586687 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fb158-365d-4d5c-8378-ecffec90159a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:07.590650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916541d0-ffe9-4c7e-a173-23650d863c69 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a58f9101f2d4785b96f0cc55be4330b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:07.592755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916541d0-ffe9-4c7e-a173-23650d863c69 service glance] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:07.592755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916541d0-ffe9-4c7e-a173-23650d863c69 service glance] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:07.592755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916541d0-ffe9-4c7e-a173-23650d863c69 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:07.601270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916541d0-ffe9-4c7e-a173-23650d863c69 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:07.602049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d873c-a214-4913-a71a-bb63b9ae2a59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:07.602677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d873c-a214-4913-a71a-bb63b9ae2a59 service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:07.602781 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916541d0-ffe9-4c7e-a173-23650d863c69 service glance] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:07.602956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d873c-a214-4913-a71a-bb63b9ae2a59 service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:07.603225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d873c-a214-4913-a71a-bb63b9ae2a59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:07.611458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1170/2369] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:55:07.661010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d873c-a214-4913-a71a-bb63b9ae2a59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:07.662645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d873c-a214-4913-a71a-bb63b9ae2a59 service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:07.673574 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1200/2370] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:55:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 18:55:17.143821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916541d0-ffe9-4c7e-a173-23650d863c69 service glance] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:17.145628 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-916541d0-ffe9-4c7e-a173-23650d863c69 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:17.225806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916541d0-ffe9-4c7e-a173-23650d863c69 service glance] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:17.231476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbb3d5-7793-446a-ba48-be2b20610e3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:17.232166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb3d5-7793-446a-ba48-be2b20610e3e service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:17.232476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb3d5-7793-446a-ba48-be2b20610e3e service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:17.232751 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb3d5-7793-446a-ba48-be2b20610e3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:17.236367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-29cbb3d5-7793-446a-ba48-be2b20610e3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:17.248008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbb3d5-7793-446a-ba48-be2b20610e3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:17.249795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbb3d5-7793-446a-ba48-be2b20610e3e service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:17.261726 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1171/2371] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:17.671412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=59 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:18.931013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d873c-a214-4913-a71a-bb63b9ae2a59 service swift] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:18.932401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3e0d873c-a214-4913-a71a-bb63b9ae2a59 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:18.989841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d873c-a214-4913-a71a-bb63b9ae2a59 service swift] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:18.993388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237d516-c752-4cab-9381-ca820b18a6e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:18.994006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237d516-c752-4cab-9381-ca820b18a6e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:18.994282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237d516-c752-4cab-9381-ca820b18a6e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:18.994544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237d516-c752-4cab-9381-ca820b18a6e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:18.996232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7237d516-c752-4cab-9381-ca820b18a6e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:19.058883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237d516-c752-4cab-9381-ca820b18a6e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:19.060308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237d516-c752-4cab-9381-ca820b18a6e6 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:19.069905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1201/2372] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:55:22.530183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8757da5-70d5-47f3-825d-0562b21a4fef None None] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:22.530702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8757da5-70d5-47f3-825d-0562b21a4fef None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:22.530862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8757da5-70d5-47f3-825d-0562b21a4fef None None] PATH_INFO: `/` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:22.532934 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1172/2373] 10.222.0.57 () {54 vars in 921 bytes} [Tue Aug 9 18:55:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:55:22.566230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ab36e-41d8-4762-a1d1-859ad29b03a9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:22.566844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ab36e-41d8-4762-a1d1-859ad29b03a9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:22.566844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ab36e-41d8-4762-a1d1-859ad29b03a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:22.587688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-521ab36e-41d8-4762-a1d1-859ad29b03a9 None None] MFA Rules not processed for user `5e0aa6b215f646c280af245a89ccf849`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:22.588917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-521ab36e-41d8-4762-a1d1-859ad29b03a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:22.654125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1202/2374] 10.222.0.57 () {58 vars in 1018 bytes} [Tue Aug 9 18:55:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:22.665790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8757da5-70d5-47f3-825d-0562b21a4fef None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:22.667592 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b8757da5-70d5-47f3-825d-0562b21a4fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:22.686269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8757da5-70d5-47f3-825d-0562b21a4fef None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:22.690015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ab36e-41d8-4762-a1d1-859ad29b03a9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:22.691488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-521ab36e-41d8-4762-a1d1-859ad29b03a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:22.692836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0c47a-407e-4670-a394-00b281ba7f22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0aa6b215f646c280af245a89ccf849', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:22.700895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0c47a-407e-4670-a394-00b281ba7f22 service placement] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:22.700895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0c47a-407e-4670-a394-00b281ba7f22 service placement] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:22.700895 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0c47a-407e-4670-a394-00b281ba7f22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:22.704690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0c47a-407e-4670-a394-00b281ba7f22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:22.706372 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0c47a-407e-4670-a394-00b281ba7f22 service placement] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:22.707104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ab36e-41d8-4762-a1d1-859ad29b03a9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:22.711794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae4331-0e66-4aa0-aa2e-89f821e58c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:22.712418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae4331-0e66-4aa0-aa2e-89f821e58c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:22.712695 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae4331-0e66-4aa0-aa2e-89f821e58c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:22.712961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae4331-0e66-4aa0-aa2e-89f821e58c4d admin admin] PATH_INFO: `/v3/users/a6e48ee30c61424ba0c82ac4fc5cd06c` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:22.714147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae4331-0e66-4aa0-aa2e-89f821e58c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6e48ee30c61424ba0c82ac4fc5cd06c)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:22.716187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae4331-0e66-4aa0-aa2e-89f821e58c4d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:22.716836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1173/2375] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:22.929274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:22.934390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5aae4331-0e66-4aa0-aa2e-89f821e58c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6e48ee30c61424ba0c82ac4fc5cd06c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:23.005792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5aae4331-0e66-4aa0-aa2e-89f821e58c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6e48ee30c61424ba0c82ac4fc5cd06c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:23.006368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5aae4331-0e66-4aa0-aa2e-89f821e58c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6e48ee30c61424ba0c82ac4fc5cd06c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:23.013915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5aae4331-0e66-4aa0-aa2e-89f821e58c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6e48ee30c61424ba0c82ac4fc5cd06c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:23.026499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5aae4331-0e66-4aa0-aa2e-89f821e58c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6e48ee30c61424ba0c82ac4fc5cd06c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:23.032456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1203/2376] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:55:22 2022] DELETE /identity/v3/users/a6e48ee30c61424ba0c82ac4fc5cd06c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:55:23.295002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0c47a-407e-4670-a394-00b281ba7f22 service placement] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:23.307298 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0c47a-407e-4670-a394-00b281ba7f22 service placement] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:23.368553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:23.393657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70161494-5da1-45fc-bdcd-e387bae3a968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:23.393657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161494-5da1-45fc-bdcd-e387bae3a968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:23.393657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161494-5da1-45fc-bdcd-e387bae3a968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:23.393657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161494-5da1-45fc-bdcd-e387bae3a968 admin admin] PATH_INFO: `/v3/projects/4d8f60ca836d43399989fb32ad78d27d` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:23.394022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70161494-5da1-45fc-bdcd-e387bae3a968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d8f60ca836d43399989fb32ad78d27d)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:23.395719 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70161494-5da1-45fc-bdcd-e387bae3a968 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:23.556962 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-70161494-5da1-45fc-bdcd-e387bae3a968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d8f60ca836d43399989fb32ad78d27d', 'request_id': 'req-70161494-5da1-45fc-bdcd-e387bae3a968'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:23.563934 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-70161494-5da1-45fc-bdcd-e387bae3a968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d8f60ca836d43399989fb32ad78d27d', 'request_id': 'req-70161494-5da1-45fc-bdcd-e387bae3a968'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:23.569917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-70161494-5da1-45fc-bdcd-e387bae3a968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d8f60ca836d43399989fb32ad78d27d', 'request_id': 'req-70161494-5da1-45fc-bdcd-e387bae3a968'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:23.663682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-70161494-5da1-45fc-bdcd-e387bae3a968 admin admin] The token cache is being invalidate because project 4d8f60ca836d43399989fb32ad78d27d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:55:23.663967 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-70161494-5da1-45fc-bdcd-e387bae3a968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:23.664638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-70161494-5da1-45fc-bdcd-e387bae3a968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:23.666252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1174/2377] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:55:23 2022] DELETE /identity/v3/projects/4d8f60ca836d43399989fb32ad78d27d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:55:23.671926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697cb66-470d-4bdf-bfd8-18274ea4b19d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:23.671926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697cb66-470d-4bdf-bfd8-18274ea4b19d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:23.671926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697cb66-470d-4bdf-bfd8-18274ea4b19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:23.693371 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8697cb66-470d-4bdf-bfd8-18274ea4b19d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:23.696198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8697cb66-470d-4bdf-bfd8-18274ea4b19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:23.758834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1204/2378] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:55:23.767168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcd639-ef67-4892-beb1-1e9d5a928142 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:23.767454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcd639-ef67-4892-beb1-1e9d5a928142 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:23.767733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcd639-ef67-4892-beb1-1e9d5a928142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:23.788231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6edcd639-ef67-4892-beb1-1e9d5a928142 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:23.789446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6edcd639-ef67-4892-beb1-1e9d5a928142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:23.797378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1175/2379] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:23.801862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697cb66-470d-4bdf-bfd8-18274ea4b19d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:23.812963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697cb66-470d-4bdf-bfd8-18274ea4b19d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:23.817895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015598ae-daa8-4161-9250-bb65a8d9f3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:23.818494 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015598ae-daa8-4161-9250-bb65a8d9f3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:23.818781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015598ae-daa8-4161-9250-bb65a8d9f3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:23.819049 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015598ae-daa8-4161-9250-bb65a8d9f3e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:23.819698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015598ae-daa8-4161-9250-bb65a8d9f3e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:55:23.820002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015598ae-daa8-4161-9250-bb65a8d9f3e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:23.821667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015598ae-daa8-4161-9250-bb65a8d9f3e8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:23.834509 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1205/2380] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:55:23.839092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcd639-ef67-4892-beb1-1e9d5a928142 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:23.849272 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcd639-ef67-4892-beb1-1e9d5a928142 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:23.854040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc38bb8-875b-4716-9f4b-43ef5d7b7d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:23.854635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc38bb8-875b-4716-9f4b-43ef5d7b7d2e admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:23.854909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc38bb8-875b-4716-9f4b-43ef5d7b7d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:23.855176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc38bb8-875b-4716-9f4b-43ef5d7b7d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:23.855808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc38bb8-875b-4716-9f4b-43ef5d7b7d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:23.857743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc38bb8-875b-4716-9f4b-43ef5d7b7d2e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:23.945402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1176/2381] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:55:23 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:23.949511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015598ae-daa8-4161-9250-bb65a8d9f3e8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:23.960045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015598ae-daa8-4161-9250-bb65a8d9f3e8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:24.015777 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fabffc-d8b8-48ed-a769-093ad6699250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:24.016914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fabffc-d8b8-48ed-a769-093ad6699250 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:24.017194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fabffc-d8b8-48ed-a769-093ad6699250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:24.017635 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fabffc-d8b8-48ed-a769-093ad6699250 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:24.018139 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fabffc-d8b8-48ed-a769-093ad6699250 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:24.020059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fabffc-d8b8-48ed-a769-093ad6699250 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:24.104419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1206/2382] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:55:23 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:24.108599 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc38bb8-875b-4716-9f4b-43ef5d7b7d2e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:24.118979 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc38bb8-875b-4716-9f4b-43ef5d7b7d2e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:24.123481 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f07b0-ffdc-43aa-bd80-c4f89b34bf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:24.124064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f07b0-ffdc-43aa-bd80-c4f89b34bf0e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:24.124341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f07b0-ffdc-43aa-bd80-c4f89b34bf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:24.124607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f07b0-ffdc-43aa-bd80-c4f89b34bf0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:24.125245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f07b0-ffdc-43aa-bd80-c4f89b34bf0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:24.126954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f07b0-ffdc-43aa-bd80-c4f89b34bf0e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:24.138731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1177/2383] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:24.143464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fabffc-d8b8-48ed-a769-093ad6699250 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:24.153387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fabffc-d8b8-48ed-a769-093ad6699250 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:24.164166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0ec28-73ae-4b31-acd7-0f5ff9d0e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:24.164776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0ec28-73ae-4b31-acd7-0f5ff9d0e471 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:24.165055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0ec28-73ae-4b31-acd7-0f5ff9d0e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:24.165324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0ec28-73ae-4b31-acd7-0f5ff9d0e471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:24.165996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0ec28-73ae-4b31-acd7-0f5ff9d0e471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:24.167608 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0ec28-73ae-4b31-acd7-0f5ff9d0e471 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:24.178619 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1207/2384] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:55:24.182984 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f07b0-ffdc-43aa-bd80-c4f89b34bf0e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:24.194935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f07b0-ffdc-43aa-bd80-c4f89b34bf0e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:24.199485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d2faa-7d3c-40ae-934a-0e06f9c5cee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:24.200127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d2faa-7d3c-40ae-934a-0e06f9c5cee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:24.200411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d2faa-7d3c-40ae-934a-0e06f9c5cee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:24.200683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d2faa-7d3c-40ae-934a-0e06f9c5cee8 admin admin] PATH_INFO: `/v3/projects/00e6ee96d5d2432abf5dc270727c7493/users/8d3f9e6b0b31451f93546fd92de2cf6b/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:24.220054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d2faa-7d3c-40ae-934a-0e06f9c5cee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e6ee96d5d2432abf5dc270727c7493, user_id=8d3f9e6b0b31451f93546fd92de2cf6b, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:24.222882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d2faa-7d3c-40ae-934a-0e06f9c5cee8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:24.322021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1178/2385] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:24 2022] PUT /identity/v3/projects/00e6ee96d5d2432abf5dc270727c7493/users/8d3f9e6b0b31451f93546fd92de2cf6b/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:24.334175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456d573-eb9b-48a1-8818-84b80a2c7a24 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:24.334461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456d573-eb9b-48a1-8818-84b80a2c7a24 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:24.334765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456d573-eb9b-48a1-8818-84b80a2c7a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:24.361490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d456d573-eb9b-48a1-8818-84b80a2c7a24 None None] MFA Rules not processed for user `8d3f9e6b0b31451f93546fd92de2cf6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:24.362640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d456d573-eb9b-48a1-8818-84b80a2c7a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:24.374324 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:24.430169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1208/2386] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:24.442869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d2faa-7d3c-40ae-934a-0e06f9c5cee8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:24.444677 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1a8d2faa-7d3c-40ae-934a-0e06f9c5cee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:24.524724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d2faa-7d3c-40ae-934a-0e06f9c5cee8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:24.530329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f89f9-5fe3-4ca5-b945-f362ad7365e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:24.530939 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f89f9-5fe3-4ca5-b945-f362ad7365e1 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:24.531218 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f89f9-5fe3-4ca5-b945-f362ad7365e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:24.531528 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f89f9-5fe3-4ca5-b945-f362ad7365e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:24.539809 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f89f9-5fe3-4ca5-b945-f362ad7365e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:24.541419 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f89f9-5fe3-4ca5-b945-f362ad7365e1 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:24.551825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1179/2387] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:27.862370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456d573-eb9b-48a1-8818-84b80a2c7a24 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:27.942310 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456d573-eb9b-48a1-8818-84b80a2c7a24 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:27.946902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:27.947785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:27.947906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:27.948169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] PATH_INFO: `/v3/users/8d3f9e6b0b31451f93546fd92de2cf6b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:27.949286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d3f9e6b0b31451f93546fd92de2cf6b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:27.951256 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:28.135002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d3f9e6b0b31451f93546fd92de2cf6b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:28.195534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d3f9e6b0b31451f93546fd92de2cf6b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:28.196078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d3f9e6b0b31451f93546fd92de2cf6b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:28.204433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d3f9e6b0b31451f93546fd92de2cf6b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:28.216346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d3f9e6b0b31451f93546fd92de2cf6b'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:28.222171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1209/2388] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:55:27 2022] DELETE /identity/v3/users/8d3f9e6b0b31451f93546fd92de2cf6b => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:28.226606 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd39e7-5b65-4310-bf5c-d9110e7a8367 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:28.227140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd39e7-5b65-4310-bf5c-d9110e7a8367 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:28.227140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd39e7-5b65-4310-bf5c-d9110e7a8367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:28.235127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:28.245948 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-06dd39e7-5b65-4310-bf5c-d9110e7a8367 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:28.247157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-06dd39e7-5b65-4310-bf5c-d9110e7a8367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:28.312279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1180/2389] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:28.322566 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:28.323966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:28.358709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:28.381212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a9453-e7ec-4249-94ba-3cf0b16c69d0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:28.384844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3bd9b-4623-4074-89e4-a215a8d8786f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:28.385434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3bd9b-4623-4074-89e4-a215a8d8786f service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:28.385743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3bd9b-4623-4074-89e4-a215a8d8786f service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:28.386018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3bd9b-4623-4074-89e4-a215a8d8786f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:28.395219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3bd9b-4623-4074-89e4-a215a8d8786f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:28.396632 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3bd9b-4623-4074-89e4-a215a8d8786f service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:28.405906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1210/2390] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:28.854071 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd39e7-5b65-4310-bf5c-d9110e7a8367 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:28.855570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-06dd39e7-5b65-4310-bf5c-d9110e7a8367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:28.868247 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd39e7-5b65-4310-bf5c-d9110e7a8367 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:28.873086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74792fb7-51ee-41dc-954a-21860a4fcf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:28.873827 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74792fb7-51ee-41dc-954a-21860a4fcf53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:28.874177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74792fb7-51ee-41dc-954a-21860a4fcf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:28.874528 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74792fb7-51ee-41dc-954a-21860a4fcf53 admin admin] PATH_INFO: `/v3/projects/00e6ee96d5d2432abf5dc270727c7493` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:28.875663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74792fb7-51ee-41dc-954a-21860a4fcf53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00e6ee96d5d2432abf5dc270727c7493)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:28.877783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74792fb7-51ee-41dc-954a-21860a4fcf53 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:28.966979 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-74792fb7-51ee-41dc-954a-21860a4fcf53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00e6ee96d5d2432abf5dc270727c7493', 'request_id': 'req-74792fb7-51ee-41dc-954a-21860a4fcf53'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:28.973940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-74792fb7-51ee-41dc-954a-21860a4fcf53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00e6ee96d5d2432abf5dc270727c7493', 'request_id': 'req-74792fb7-51ee-41dc-954a-21860a4fcf53'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:28.979571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-74792fb7-51ee-41dc-954a-21860a4fcf53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00e6ee96d5d2432abf5dc270727c7493', 'request_id': 'req-74792fb7-51ee-41dc-954a-21860a4fcf53'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:29.051893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-74792fb7-51ee-41dc-954a-21860a4fcf53 admin admin] The token cache is being invalidate because project 00e6ee96d5d2432abf5dc270727c7493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:55:29.052172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-74792fb7-51ee-41dc-954a-21860a4fcf53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:29.052793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-74792fb7-51ee-41dc-954a-21860a4fcf53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:29.054312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1181/2391] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:55:28 2022] DELETE /identity/v3/projects/00e6ee96d5d2432abf5dc270727c7493 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:29.059465 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e78ae-a628-4fae-9cde-649bfdff6c99 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:29.059971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e78ae-a628-4fae-9cde-649bfdff6c99 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:29.060109 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e78ae-a628-4fae-9cde-649bfdff6c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:29.080190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c86e78ae-a628-4fae-9cde-649bfdff6c99 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:29.081393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c86e78ae-a628-4fae-9cde-649bfdff6c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:29.140380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1211/2392] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:55:29.148753 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bd54b-8d5f-46d6-8741-4d897293a410 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:29.149032 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bd54b-8d5f-46d6-8741-4d897293a410 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:29.149300 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bd54b-8d5f-46d6-8741-4d897293a410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:29.169887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-cb8bd54b-8d5f-46d6-8741-4d897293a410 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:29.171088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cb8bd54b-8d5f-46d6-8741-4d897293a410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:29.178912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1182/2393] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:29.184427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e78ae-a628-4fae-9cde-649bfdff6c99 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:29.194868 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e78ae-a628-4fae-9cde-649bfdff6c99 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:29.199401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d599251-e720-4e27-a045-22a7c2b1388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:29.200039 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d599251-e720-4e27-a045-22a7c2b1388b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:29.200316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d599251-e720-4e27-a045-22a7c2b1388b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:29.200584 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d599251-e720-4e27-a045-22a7c2b1388b admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:29.201233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d599251-e720-4e27-a045-22a7c2b1388b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:55:29.201547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d599251-e720-4e27-a045-22a7c2b1388b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:29.203161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d599251-e720-4e27-a045-22a7c2b1388b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:29.215640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1212/2394] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:55:29.221505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bd54b-8d5f-46d6-8741-4d897293a410 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:29.230889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bd54b-8d5f-46d6-8741-4d897293a410 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:29.235995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e120928-8da2-49f0-902e-d202a18f312f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:29.236723 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e120928-8da2-49f0-902e-d202a18f312f admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:29.237007 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e120928-8da2-49f0-902e-d202a18f312f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:29.237282 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e120928-8da2-49f0-902e-d202a18f312f admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:29.237958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e120928-8da2-49f0-902e-d202a18f312f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:29.239896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e120928-8da2-49f0-902e-d202a18f312f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:29.340789 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1183/2395] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:55:29 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:55:29.345702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d599251-e720-4e27-a045-22a7c2b1388b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:29.356067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d599251-e720-4e27-a045-22a7c2b1388b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:29.412238 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f878f49-57db-41f5-9e81-26c4ddd00c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:29.412769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f878f49-57db-41f5-9e81-26c4ddd00c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:29.413051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f878f49-57db-41f5-9e81-26c4ddd00c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:29.413333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f878f49-57db-41f5-9e81-26c4ddd00c34 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:29.413984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f878f49-57db-41f5-9e81-26c4ddd00c34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:29.415916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f878f49-57db-41f5-9e81-26c4ddd00c34 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:29.481055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1213/2396] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:55:29 2022] POST /identity/v3/users => generated 283 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:55:29.485144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e120928-8da2-49f0-902e-d202a18f312f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:29.495377 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e120928-8da2-49f0-902e-d202a18f312f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:29.500021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538a2ef-a9af-4ecf-913f-c1d090e9fbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:29.501140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538a2ef-a9af-4ecf-913f-c1d090e9fbbd admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:29.501418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538a2ef-a9af-4ecf-913f-c1d090e9fbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:29.501721 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538a2ef-a9af-4ecf-913f-c1d090e9fbbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:29.502357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538a2ef-a9af-4ecf-913f-c1d090e9fbbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:29.503988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538a2ef-a9af-4ecf-913f-c1d090e9fbbd admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:29.514267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1184/2397] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:29.518898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f878f49-57db-41f5-9e81-26c4ddd00c34 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:29.529445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f878f49-57db-41f5-9e81-26c4ddd00c34 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:29.534186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602491f-d40b-492a-a288-40bf903238ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:29.534788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602491f-d40b-492a-a288-40bf903238ee admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:29.535073 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602491f-d40b-492a-a288-40bf903238ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:29.535352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602491f-d40b-492a-a288-40bf903238ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:29.536003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602491f-d40b-492a-a288-40bf903238ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:29.537634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602491f-d40b-492a-a288-40bf903238ee admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:29.549226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1214/2398] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:29.553511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538a2ef-a9af-4ecf-913f-c1d090e9fbbd admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:29.564120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538a2ef-a9af-4ecf-913f-c1d090e9fbbd admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:29.568879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edfda3-3a07-41aa-b69f-a92a43891db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:29.569541 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edfda3-3a07-41aa-b69f-a92a43891db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:29.569863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edfda3-3a07-41aa-b69f-a92a43891db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:29.570137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edfda3-3a07-41aa-b69f-a92a43891db9 admin admin] PATH_INFO: `/v3/projects/b1e1bb7483f443f3b26fdb24ef564cc7/users/153d5c971b70429bb73279fa4173cbc7/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:29.585825 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edfda3-3a07-41aa-b69f-a92a43891db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1e1bb7483f443f3b26fdb24ef564cc7, user_id=153d5c971b70429bb73279fa4173cbc7, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:29.588501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edfda3-3a07-41aa-b69f-a92a43891db9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:29.647103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1185/2399] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:29 2022] PUT /identity/v3/projects/b1e1bb7483f443f3b26fdb24ef564cc7/users/153d5c971b70429bb73279fa4173cbc7/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:29.657450 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723d98a-f660-4244-8d69-bb3b2db6bbc7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:29.657765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723d98a-f660-4244-8d69-bb3b2db6bbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:29.658037 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723d98a-f660-4244-8d69-bb3b2db6bbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:29.685652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e723d98a-f660-4244-8d69-bb3b2db6bbc7 None None] MFA Rules not processed for user `153d5c971b70429bb73279fa4173cbc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:29.686779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e723d98a-f660-4244-8d69-bb3b2db6bbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:29.717460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:29.752180 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1215/2400] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:29.759996 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edfda3-3a07-41aa-b69f-a92a43891db9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:29.787272 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:29.822587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edfda3-3a07-41aa-b69f-a92a43891db9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:29.827937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f3b6b-d384-4017-8cb2-7ce831df492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:29.828552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f3b6b-d384-4017-8cb2-7ce831df492e admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:29.828839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f3b6b-d384-4017-8cb2-7ce831df492e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:29.829112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f3b6b-d384-4017-8cb2-7ce831df492e admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:29.829804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f3b6b-d384-4017-8cb2-7ce831df492e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:29.831705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f3b6b-d384-4017-8cb2-7ce831df492e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:29.916484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1186/2401] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:55:29 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:29.920560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723d98a-f660-4244-8d69-bb3b2db6bbc7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:29.931569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723d98a-f660-4244-8d69-bb3b2db6bbc7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:29.992093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77e5d2-1698-416c-a690-beb87b8bf6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:29.992693 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77e5d2-1698-416c-a690-beb87b8bf6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:29.992971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77e5d2-1698-416c-a690-beb87b8bf6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:29.993240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77e5d2-1698-416c-a690-beb87b8bf6d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:29.993895 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77e5d2-1698-416c-a690-beb87b8bf6d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:29.995799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77e5d2-1698-416c-a690-beb87b8bf6d4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:30.076908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1216/2402] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:55:29 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:55:30.081076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f3b6b-d384-4017-8cb2-7ce831df492e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:30.091274 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f3b6b-d384-4017-8cb2-7ce831df492e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:30.095906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a76cd-ca5a-4a54-8fc8-27cde9425774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:30.096491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a76cd-ca5a-4a54-8fc8-27cde9425774 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:30.096767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a76cd-ca5a-4a54-8fc8-27cde9425774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:30.097036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a76cd-ca5a-4a54-8fc8-27cde9425774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:30.097717 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a76cd-ca5a-4a54-8fc8-27cde9425774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:30.099298 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a76cd-ca5a-4a54-8fc8-27cde9425774 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:30.110375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1187/2403] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:30.114549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77e5d2-1698-416c-a690-beb87b8bf6d4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:30.124438 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77e5d2-1698-416c-a690-beb87b8bf6d4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:30.129833 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22089aab-e0db-4ec3-9c51-fb68adc7a154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:30.130474 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22089aab-e0db-4ec3-9c51-fb68adc7a154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:30.130753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22089aab-e0db-4ec3-9c51-fb68adc7a154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:30.131023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22089aab-e0db-4ec3-9c51-fb68adc7a154 admin admin] PATH_INFO: `/v3/projects/89643456dfd64dff8c94db0dfa5f8bcc/users/2d83b3143944430894ac6ae4e3db3b61/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:30.146624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22089aab-e0db-4ec3-9c51-fb68adc7a154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89643456dfd64dff8c94db0dfa5f8bcc, user_id=2d83b3143944430894ac6ae4e3db3b61, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:30.149286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22089aab-e0db-4ec3-9c51-fb68adc7a154 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:30.224587 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1217/2404] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:30 2022] PUT /identity/v3/projects/89643456dfd64dff8c94db0dfa5f8bcc/users/2d83b3143944430894ac6ae4e3db3b61/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:55:30.228716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a76cd-ca5a-4a54-8fc8-27cde9425774 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:30.239565 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a76cd-ca5a-4a54-8fc8-27cde9425774 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:30.300217 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f170b-2b52-46bf-886f-e44baaf0ae2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:30.300807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f170b-2b52-46bf-886f-e44baaf0ae2d admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:30.301083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f170b-2b52-46bf-886f-e44baaf0ae2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:30.301349 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f170b-2b52-46bf-886f-e44baaf0ae2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:30.302020 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f170b-2b52-46bf-886f-e44baaf0ae2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:30.303597 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f170b-2b52-46bf-886f-e44baaf0ae2d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:30.313491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1188/2405] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:30.317720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22089aab-e0db-4ec3-9c51-fb68adc7a154 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:30.328041 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22089aab-e0db-4ec3-9c51-fb68adc7a154 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:30.332550 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a058d4-afa3-4e90-badc-ff040cb789c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:30.333213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a058d4-afa3-4e90-badc-ff040cb789c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:30.333489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a058d4-afa3-4e90-badc-ff040cb789c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:30.333789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a058d4-afa3-4e90-badc-ff040cb789c2 admin admin] PATH_INFO: `/v3/projects/89643456dfd64dff8c94db0dfa5f8bcc/users/2d83b3143944430894ac6ae4e3db3b61/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:30.335656 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a058d4-afa3-4e90-badc-ff040cb789c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89643456dfd64dff8c94db0dfa5f8bcc, user_id=2d83b3143944430894ac6ae4e3db3b61, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:30.338343 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a058d4-afa3-4e90-badc-ff040cb789c2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:30.349316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1218/2406] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:30 2022] PUT /identity/v3/projects/89643456dfd64dff8c94db0dfa5f8bcc/users/2d83b3143944430894ac6ae4e3db3b61/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:30.361613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd8ca2-2f51-4c61-b90a-ea5b10ef0032 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:30.361902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd8ca2-2f51-4c61-b90a-ea5b10ef0032 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:30.362181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd8ca2-2f51-4c61-b90a-ea5b10ef0032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:30.390046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-28dd8ca2-2f51-4c61-b90a-ea5b10ef0032 None None] MFA Rules not processed for user `2d83b3143944430894ac6ae4e3db3b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:30.395430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-28dd8ca2-2f51-4c61-b90a-ea5b10ef0032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:30.464071 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1189/2407] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:30.476417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a058d4-afa3-4e90-badc-ff040cb789c2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:30.478079 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-41a058d4-afa3-4e90-badc-ff040cb789c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:30.535722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a058d4-afa3-4e90-badc-ff040cb789c2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:30.539508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d790f-8dba-4947-a22e-5263de40615f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:30.540098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d790f-8dba-4947-a22e-5263de40615f service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:30.540373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d790f-8dba-4947-a22e-5263de40615f service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:30.540640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d790f-8dba-4947-a22e-5263de40615f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:30.550272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d790f-8dba-4947-a22e-5263de40615f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:30.551703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d790f-8dba-4947-a22e-5263de40615f service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:30.565212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1219/2408] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:31.613107 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd8ca2-2f51-4c61-b90a-ea5b10ef0032 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:31.636794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:31.679920 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd8ca2-2f51-4c61-b90a-ea5b10ef0032 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:31.685208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:31.686251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:31.686735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:31.687251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] PATH_INFO: `/v3/users/153d5c971b70429bb73279fa4173cbc7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:31.688847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=153d5c971b70429bb73279fa4173cbc7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:31.691086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:31.900387 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '153d5c971b70429bb73279fa4173cbc7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:31.972177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '153d5c971b70429bb73279fa4173cbc7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:31.972942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '153d5c971b70429bb73279fa4173cbc7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:31.980538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '153d5c971b70429bb73279fa4173cbc7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:31.993243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '153d5c971b70429bb73279fa4173cbc7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:31.999183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1190/2409] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:55:31 2022] DELETE /identity/v3/users/153d5c971b70429bb73279fa4173cbc7 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:32.011664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156ce87-cc70-4ee2-9224-7bac89fda1a2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:32.012611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156ce87-cc70-4ee2-9224-7bac89fda1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:32.012890 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156ce87-cc70-4ee2-9224-7bac89fda1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:32.032616 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d156ce87-cc70-4ee2-9224-7bac89fda1a2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:32.033842 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d156ce87-cc70-4ee2-9224-7bac89fda1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:32.057428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:32.101241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1220/2410] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:32.111867 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:32.113271 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:32.172031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86a55d-9a72-4553-97b3-007bfd4335cc admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:32.175741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16058a41-852f-4a05-aee5-e05d8d5f2d83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:32.176346 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16058a41-852f-4a05-aee5-e05d8d5f2d83 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:32.176620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16058a41-852f-4a05-aee5-e05d8d5f2d83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:32.177197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16058a41-852f-4a05-aee5-e05d8d5f2d83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:32.188494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16058a41-852f-4a05-aee5-e05d8d5f2d83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:32.189959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16058a41-852f-4a05-aee5-e05d8d5f2d83 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:32.199858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1191/2411] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:32.623906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156ce87-cc70-4ee2-9224-7bac89fda1a2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:32.623906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d156ce87-cc70-4ee2-9224-7bac89fda1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:32.632963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156ce87-cc70-4ee2-9224-7bac89fda1a2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:32.637649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:32.638274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:32.638548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:32.638815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] PATH_INFO: `/v3/projects/b1e1bb7483f443f3b26fdb24ef564cc7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:32.639967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1e1bb7483f443f3b26fdb24ef564cc7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:32.642698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:32.774266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1e1bb7483f443f3b26fdb24ef564cc7', 'request_id': 'req-bcb56762-3983-4fc5-a947-67753619dcb6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:32.779998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1e1bb7483f443f3b26fdb24ef564cc7', 'request_id': 'req-bcb56762-3983-4fc5-a947-67753619dcb6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:32.786503 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1e1bb7483f443f3b26fdb24ef564cc7', 'request_id': 'req-bcb56762-3983-4fc5-a947-67753619dcb6'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:32.876487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] The token cache is being invalidate because project b1e1bb7483f443f3b26fdb24ef564cc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:55:32.877401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:32.878097 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:32.880688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1221/2412] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:55:32 2022] DELETE /identity/v3/projects/b1e1bb7483f443f3b26fdb24ef564cc7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:32.884683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16058a41-852f-4a05-aee5-e05d8d5f2d83 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:32.895067 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16058a41-852f-4a05-aee5-e05d8d5f2d83 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:32.940577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:32.956424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d09c8-1527-4815-adaa-b98d4d1cd79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:32.957091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d09c8-1527-4815-adaa-b98d4d1cd79e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:32.957367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d09c8-1527-4815-adaa-b98d4d1cd79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:32.957669 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d09c8-1527-4815-adaa-b98d4d1cd79e admin admin] PATH_INFO: `/v3/users/2d83b3143944430894ac6ae4e3db3b61` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:32.958803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d09c8-1527-4815-adaa-b98d4d1cd79e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d83b3143944430894ac6ae4e3db3b61)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:32.961128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d09c8-1527-4815-adaa-b98d4d1cd79e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:33.100048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c96d09c8-1527-4815-adaa-b98d4d1cd79e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d83b3143944430894ac6ae4e3db3b61'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:33.184646 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c96d09c8-1527-4815-adaa-b98d4d1cd79e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d83b3143944430894ac6ae4e3db3b61'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:33.185291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c96d09c8-1527-4815-adaa-b98d4d1cd79e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d83b3143944430894ac6ae4e3db3b61'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:33.193139 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c96d09c8-1527-4815-adaa-b98d4d1cd79e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d83b3143944430894ac6ae4e3db3b61'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:33.204923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c96d09c8-1527-4815-adaa-b98d4d1cd79e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d83b3143944430894ac6ae4e3db3b61'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:33.211047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1192/2413] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:55:32 2022] DELETE /identity/v3/users/2d83b3143944430894ac6ae4e3db3b61 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:33.636755 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:33.642616 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:33.652752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb56762-3983-4fc5-a947-67753619dcb6 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:33.711973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ade8e-6551-47f0-8cd4-b43d13039385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:33.712510 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ade8e-6551-47f0-8cd4-b43d13039385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:33.712791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ade8e-6551-47f0-8cd4-b43d13039385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:33.713066 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ade8e-6551-47f0-8cd4-b43d13039385 admin admin] PATH_INFO: `/v3/projects/89643456dfd64dff8c94db0dfa5f8bcc` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:33.714170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ade8e-6551-47f0-8cd4-b43d13039385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89643456dfd64dff8c94db0dfa5f8bcc)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:33.716126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ade8e-6551-47f0-8cd4-b43d13039385 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:33.840522 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d60ade8e-6551-47f0-8cd4-b43d13039385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89643456dfd64dff8c94db0dfa5f8bcc', 'request_id': 'req-d60ade8e-6551-47f0-8cd4-b43d13039385'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:33.842463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:33.846935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d60ade8e-6551-47f0-8cd4-b43d13039385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89643456dfd64dff8c94db0dfa5f8bcc', 'request_id': 'req-d60ade8e-6551-47f0-8cd4-b43d13039385'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:33.853112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d60ade8e-6551-47f0-8cd4-b43d13039385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89643456dfd64dff8c94db0dfa5f8bcc', 'request_id': 'req-d60ade8e-6551-47f0-8cd4-b43d13039385'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:33.948294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d60ade8e-6551-47f0-8cd4-b43d13039385 admin admin] The token cache is being invalidate because project 89643456dfd64dff8c94db0dfa5f8bcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:55:33.948586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d60ade8e-6551-47f0-8cd4-b43d13039385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:33.949221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d60ade8e-6551-47f0-8cd4-b43d13039385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:33.950758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1222/2414] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:55:33 2022] DELETE /identity/v3/projects/89643456dfd64dff8c94db0dfa5f8bcc => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:33.956393 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d383df-9b5e-483d-9201-1c73512e339c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:33.958186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d383df-9b5e-483d-9201-1c73512e339c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:33.958186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d383df-9b5e-483d-9201-1c73512e339c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:33.976608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b1d383df-9b5e-483d-9201-1c73512e339c None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:33.977775 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b1d383df-9b5e-483d-9201-1c73512e339c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:34.038086 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1193/2415] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:34.045657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d40b6-c543-4743-8d73-54074efbd786 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:34.045945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d40b6-c543-4743-8d73-54074efbd786 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:34.046222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d40b6-c543-4743-8d73-54074efbd786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:34.067272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-885d40b6-c543-4743-8d73-54074efbd786 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:34.068488 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-885d40b6-c543-4743-8d73-54074efbd786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:34.076427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1223/2416] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:34.084522 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d383df-9b5e-483d-9201-1c73512e339c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:34.094429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d383df-9b5e-483d-9201-1c73512e339c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:34.099186 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0ebb0-dde8-4f4e-a9cf-fdae7c04df16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:34.099783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0ebb0-dde8-4f4e-a9cf-fdae7c04df16 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:34.100059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0ebb0-dde8-4f4e-a9cf-fdae7c04df16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:34.100329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0ebb0-dde8-4f4e-a9cf-fdae7c04df16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:34.100974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0ebb0-dde8-4f4e-a9cf-fdae7c04df16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:55:34.101277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0ebb0-dde8-4f4e-a9cf-fdae7c04df16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:34.102958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0ebb0-dde8-4f4e-a9cf-fdae7c04df16 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:34.114999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1194/2417] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:55:34.119284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d40b6-c543-4743-8d73-54074efbd786 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:34.128960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d40b6-c543-4743-8d73-54074efbd786 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:34.133663 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4886d5-8625-4b22-9739-aa3f0aab614f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:34.134252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4886d5-8625-4b22-9739-aa3f0aab614f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:34.135784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4886d5-8625-4b22-9739-aa3f0aab614f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:34.135784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4886d5-8625-4b22-9739-aa3f0aab614f admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:34.135784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4886d5-8625-4b22-9739-aa3f0aab614f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:34.137280 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4886d5-8625-4b22-9739-aa3f0aab614f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:34.214367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1224/2418] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:55:34 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:34.218544 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0ebb0-dde8-4f4e-a9cf-fdae7c04df16 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:34.228521 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0ebb0-dde8-4f4e-a9cf-fdae7c04df16 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:34.274332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:34.286198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b50961-9c28-4a7f-98d6-3e08a74d4853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:34.286805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b50961-9c28-4a7f-98d6-3e08a74d4853 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:34.287089 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b50961-9c28-4a7f-98d6-3e08a74d4853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:34.287362 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b50961-9c28-4a7f-98d6-3e08a74d4853 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:34.288045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b50961-9c28-4a7f-98d6-3e08a74d4853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:34.290001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b50961-9c28-4a7f-98d6-3e08a74d4853 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:34.367807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1195/2419] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:55:34 2022] POST /identity/v3/users => generated 282 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:34.372044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4886d5-8625-4b22-9739-aa3f0aab614f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:34.383010 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4886d5-8625-4b22-9739-aa3f0aab614f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:34.387552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9f002-b3a6-4300-80e3-ef50e238231a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:34.388140 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9f002-b3a6-4300-80e3-ef50e238231a admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:34.388470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9f002-b3a6-4300-80e3-ef50e238231a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:34.388772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9f002-b3a6-4300-80e3-ef50e238231a admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:34.389398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9f002-b3a6-4300-80e3-ef50e238231a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:34.391093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9f002-b3a6-4300-80e3-ef50e238231a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:34.401728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1225/2420] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:34.405654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b50961-9c28-4a7f-98d6-3e08a74d4853 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:34.415517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b50961-9c28-4a7f-98d6-3e08a74d4853 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:34.420458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f52aab-8a99-4300-86f3-3007d7f0f34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:34.421057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f52aab-8a99-4300-86f3-3007d7f0f34b admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:34.421340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f52aab-8a99-4300-86f3-3007d7f0f34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:34.421655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f52aab-8a99-4300-86f3-3007d7f0f34b admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:34.422811 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f52aab-8a99-4300-86f3-3007d7f0f34b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:34.424517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f52aab-8a99-4300-86f3-3007d7f0f34b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:34.435676 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1196/2421] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:34.440427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9f002-b3a6-4300-80e3-ef50e238231a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:34.450363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9f002-b3a6-4300-80e3-ef50e238231a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:34.455019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe892cf-ce61-4b12-802d-702114c6d164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:34.455660 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe892cf-ce61-4b12-802d-702114c6d164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:34.455934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe892cf-ce61-4b12-802d-702114c6d164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:34.456208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe892cf-ce61-4b12-802d-702114c6d164 admin admin] PATH_INFO: `/v3/projects/d5d4703971b4479e8284c7bd7cbb028e/users/95e4f0998d5d4fb19e0f5f1d7ca0e69c/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:34.470577 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe892cf-ce61-4b12-802d-702114c6d164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d4703971b4479e8284c7bd7cbb028e, user_id=95e4f0998d5d4fb19e0f5f1d7ca0e69c, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:34.473228 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe892cf-ce61-4b12-802d-702114c6d164 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:34.578260 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1226/2422] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:34 2022] PUT /identity/v3/projects/d5d4703971b4479e8284c7bd7cbb028e/users/95e4f0998d5d4fb19e0f5f1d7ca0e69c/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:55:34.588786 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa13f78-9357-4506-b580-7b30cfc21163 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:34.591021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa13f78-9357-4506-b580-7b30cfc21163 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:34.591021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa13f78-9357-4506-b580-7b30cfc21163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:34.626320 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9fa13f78-9357-4506-b580-7b30cfc21163 None None] MFA Rules not processed for user `95e4f0998d5d4fb19e0f5f1d7ca0e69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:34.626320 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9fa13f78-9357-4506-b580-7b30cfc21163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:34.691734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1197/2423] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:34.696340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe892cf-ce61-4b12-802d-702114c6d164 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:34.760495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe892cf-ce61-4b12-802d-702114c6d164 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:34.765401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553aceb-ef7d-4577-8b5b-948b6168343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:34.770461 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553aceb-ef7d-4577-8b5b-948b6168343c admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:34.775434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553aceb-ef7d-4577-8b5b-948b6168343c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:34.775434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553aceb-ef7d-4577-8b5b-948b6168343c admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:34.775434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553aceb-ef7d-4577-8b5b-948b6168343c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:34.775434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553aceb-ef7d-4577-8b5b-948b6168343c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:34.867185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1227/2424] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:55:34 2022] POST /identity/v3/projects => generated 363 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:34.877146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa13f78-9357-4506-b580-7b30cfc21163 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:34.882447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa13f78-9357-4506-b580-7b30cfc21163 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:34.942607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc045058-9e0a-48ba-b1a4-31ab94f64c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:34.943219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc045058-9e0a-48ba-b1a4-31ab94f64c2e admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:34.943500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc045058-9e0a-48ba-b1a4-31ab94f64c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:34.943780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc045058-9e0a-48ba-b1a4-31ab94f64c2e admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:34.944422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc045058-9e0a-48ba-b1a4-31ab94f64c2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:34.946411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc045058-9e0a-48ba-b1a4-31ab94f64c2e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:35.035904 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1198/2425] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:55:34 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:35.040067 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553aceb-ef7d-4577-8b5b-948b6168343c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:35.050991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553aceb-ef7d-4577-8b5b-948b6168343c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:35.055565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ceda2-d3a3-4f92-855c-bd345eb1bdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:35.056179 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ceda2-d3a3-4f92-855c-bd345eb1bdcc admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:35.056455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ceda2-d3a3-4f92-855c-bd345eb1bdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:35.056727 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ceda2-d3a3-4f92-855c-bd345eb1bdcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:35.057356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ceda2-d3a3-4f92-855c-bd345eb1bdcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:35.059234 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ceda2-d3a3-4f92-855c-bd345eb1bdcc admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:35.070358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1228/2426] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:35.075664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc045058-9e0a-48ba-b1a4-31ab94f64c2e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:35.086105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc045058-9e0a-48ba-b1a4-31ab94f64c2e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:35.090862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455da82e-dc19-472b-9d04-608127ff64c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:35.091503 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455da82e-dc19-472b-9d04-608127ff64c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:35.091781 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455da82e-dc19-472b-9d04-608127ff64c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:35.092054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455da82e-dc19-472b-9d04-608127ff64c6 admin admin] PATH_INFO: `/v3/projects/9c29d1eff3fd484c810ba4c34128a3c3/users/fa7f53bff12a42fe9b818617293c6981/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:35.108452 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455da82e-dc19-472b-9d04-608127ff64c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c29d1eff3fd484c810ba4c34128a3c3, user_id=fa7f53bff12a42fe9b818617293c6981, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:35.111649 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455da82e-dc19-472b-9d04-608127ff64c6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:35.202726 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1199/2427] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:35 2022] PUT /identity/v3/projects/9c29d1eff3fd484c810ba4c34128a3c3/users/fa7f53bff12a42fe9b818617293c6981/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:55:35.206711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ceda2-d3a3-4f92-855c-bd345eb1bdcc admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:35.216787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ceda2-d3a3-4f92-855c-bd345eb1bdcc admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:35.271196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:35.281328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fea36-ee5b-4304-ad25-4d62e587202b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:35.282009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fea36-ee5b-4304-ad25-4d62e587202b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:35.282298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fea36-ee5b-4304-ad25-4d62e587202b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:35.282548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fea36-ee5b-4304-ad25-4d62e587202b admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:35.283248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fea36-ee5b-4304-ad25-4d62e587202b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:35.284849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fea36-ee5b-4304-ad25-4d62e587202b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:35.294825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1229/2428] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:35.299801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455da82e-dc19-472b-9d04-608127ff64c6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:35.310252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455da82e-dc19-472b-9d04-608127ff64c6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:35.315431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822d9f8-5aa2-4ff1-9d7a-dac056387b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:35.316077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822d9f8-5aa2-4ff1-9d7a-dac056387b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:35.316356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822d9f8-5aa2-4ff1-9d7a-dac056387b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:35.316627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822d9f8-5aa2-4ff1-9d7a-dac056387b63 admin admin] PATH_INFO: `/v3/projects/9c29d1eff3fd484c810ba4c34128a3c3/users/fa7f53bff12a42fe9b818617293c6981/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:35.318618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822d9f8-5aa2-4ff1-9d7a-dac056387b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c29d1eff3fd484c810ba4c34128a3c3, user_id=fa7f53bff12a42fe9b818617293c6981, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:35.321887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822d9f8-5aa2-4ff1-9d7a-dac056387b63 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:35.332454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1200/2429] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:35 2022] PUT /identity/v3/projects/9c29d1eff3fd484c810ba4c34128a3c3/users/fa7f53bff12a42fe9b818617293c6981/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:55:35.344053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e566e4-7c54-40eb-8013-3385dff73628 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:35.344284 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e566e4-7c54-40eb-8013-3385dff73628 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:35.344558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e566e4-7c54-40eb-8013-3385dff73628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:35.373886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-30e566e4-7c54-40eb-8013-3385dff73628 None None] MFA Rules not processed for user `fa7f53bff12a42fe9b818617293c6981`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:35.375105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-30e566e4-7c54-40eb-8013-3385dff73628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:35.447750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1230/2430] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:55:35.459202 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822d9f8-5aa2-4ff1-9d7a-dac056387b63 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:35.460881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c822d9f8-5aa2-4ff1-9d7a-dac056387b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:35.474272 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:35.519172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822d9f8-5aa2-4ff1-9d7a-dac056387b63 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:35.522768 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cedc7-563f-4a61-90fd-8264c4189b4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:35.523359 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cedc7-563f-4a61-90fd-8264c4189b4c service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:35.523634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cedc7-563f-4a61-90fd-8264c4189b4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:35.523901 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cedc7-563f-4a61-90fd-8264c4189b4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:35.534392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421cedc7-563f-4a61-90fd-8264c4189b4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:35.535967 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421cedc7-563f-4a61-90fd-8264c4189b4c service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:35.545775 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1201/2431] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:35.694799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e566e4-7c54-40eb-8013-3385dff73628 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:35.704671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e566e4-7c54-40eb-8013-3385dff73628 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:35.708496 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43637e-828c-47b2-a4ea-5895f75b4233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:35.709102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43637e-828c-47b2-a4ea-5895f75b4233 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:35.709378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43637e-828c-47b2-a4ea-5895f75b4233 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:35.709687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43637e-828c-47b2-a4ea-5895f75b4233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:35.767720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43637e-828c-47b2-a4ea-5895f75b4233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:35.769196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43637e-828c-47b2-a4ea-5895f75b4233 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:35.778061 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1231/2432] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:55:37.195069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cedc7-563f-4a61-90fd-8264c4189b4c service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:37.196531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-421cedc7-563f-4a61-90fd-8264c4189b4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:37.269933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cedc7-563f-4a61-90fd-8264c4189b4c service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:37.275120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:37.275762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:37.276038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:37.276620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] PATH_INFO: `/v3/users/4754b4dd80e244be9e15121d0ebfefe1` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:37.277455 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4754b4dd80e244be9e15121d0ebfefe1)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:37.279409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:37.472629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4754b4dd80e244be9e15121d0ebfefe1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:37.537608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4754b4dd80e244be9e15121d0ebfefe1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:37.538270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4754b4dd80e244be9e15121d0ebfefe1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:37.545865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4754b4dd80e244be9e15121d0ebfefe1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:37.557496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4754b4dd80e244be9e15121d0ebfefe1'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:37.564058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1202/2433] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:55:37 2022] DELETE /identity/v3/users/4754b4dd80e244be9e15121d0ebfefe1 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:37.569424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb2265-8488-4155-ab7b-dbd33c8f0039 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:37.569975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb2265-8488-4155-ab7b-dbd33c8f0039 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:37.570096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb2265-8488-4155-ab7b-dbd33c8f0039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:37.589863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-cbcb2265-8488-4155-ab7b-dbd33c8f0039 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:37.591065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cbcb2265-8488-4155-ab7b-dbd33c8f0039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:37.655300 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1232/2434] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:55:37.666557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:37.668743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:37.716739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:37.740420 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c955394-8653-4d3d-966e-f63a14e8a394 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:37.744424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a9f8f-8bcd-4e9e-8e1b-e54ca966d34c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:37.745032 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a9f8f-8bcd-4e9e-8e1b-e54ca966d34c service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:37.745314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a9f8f-8bcd-4e9e-8e1b-e54ca966d34c service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:37.745613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a9f8f-8bcd-4e9e-8e1b-e54ca966d34c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:37.756936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a9f8f-8bcd-4e9e-8e1b-e54ca966d34c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:37.758418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a9f8f-8bcd-4e9e-8e1b-e54ca966d34c service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:37.768845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1203/2435] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:38.438583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb2265-8488-4155-ab7b-dbd33c8f0039 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:38.440744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cbcb2265-8488-4155-ab7b-dbd33c8f0039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:38.456305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb2265-8488-4155-ab7b-dbd33c8f0039 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:38.461991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2536f6-7381-41f6-a0cf-65422a3e9131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:38.462640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2536f6-7381-41f6-a0cf-65422a3e9131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:38.463056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2536f6-7381-41f6-a0cf-65422a3e9131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:38.463331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2536f6-7381-41f6-a0cf-65422a3e9131 admin admin] PATH_INFO: `/v3/projects/1a178a776de44a638b9cb903a597f789` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:38.464468 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2536f6-7381-41f6-a0cf-65422a3e9131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a178a776de44a638b9cb903a597f789)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:38.467041 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2536f6-7381-41f6-a0cf-65422a3e9131 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:38.475239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:38.602636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9e2536f6-7381-41f6-a0cf-65422a3e9131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a178a776de44a638b9cb903a597f789', 'request_id': 'req-9e2536f6-7381-41f6-a0cf-65422a3e9131'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:38.608286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9e2536f6-7381-41f6-a0cf-65422a3e9131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a178a776de44a638b9cb903a597f789', 'request_id': 'req-9e2536f6-7381-41f6-a0cf-65422a3e9131'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:38.614476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9e2536f6-7381-41f6-a0cf-65422a3e9131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a178a776de44a638b9cb903a597f789', 'request_id': 'req-9e2536f6-7381-41f6-a0cf-65422a3e9131'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:38.680249 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9e2536f6-7381-41f6-a0cf-65422a3e9131 admin admin] The token cache is being invalidate because project 1a178a776de44a638b9cb903a597f789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:55:38.680544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9e2536f6-7381-41f6-a0cf-65422a3e9131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:38.681187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-9e2536f6-7381-41f6-a0cf-65422a3e9131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:38.682729 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1233/2436] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:55:38 2022] DELETE /identity/v3/projects/1a178a776de44a638b9cb903a597f789 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:38.688051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d939a9f-b830-48f2-b32f-fb0f9efc415d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:38.691251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d939a9f-b830-48f2-b32f-fb0f9efc415d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:38.691251 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d939a9f-b830-48f2-b32f-fb0f9efc415d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:38.709456 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-8d939a9f-b830-48f2-b32f-fb0f9efc415d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:38.710625 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8d939a9f-b830-48f2-b32f-fb0f9efc415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:38.771142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1204/2437] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:55:38.778956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0763e-a776-4344-85cf-306db7c5ede3 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:38.779236 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0763e-a776-4344-85cf-306db7c5ede3 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:38.779508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0763e-a776-4344-85cf-306db7c5ede3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:38.800711 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-76d0763e-a776-4344-85cf-306db7c5ede3 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:38.801954 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-76d0763e-a776-4344-85cf-306db7c5ede3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:38.810133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1234/2438] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:38.815206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d939a9f-b830-48f2-b32f-fb0f9efc415d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:38.825180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d939a9f-b830-48f2-b32f-fb0f9efc415d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:38.829801 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63933205-75e5-4335-b4a8-cc2bb171a614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:38.830394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933205-75e5-4335-b4a8-cc2bb171a614 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:38.830670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933205-75e5-4335-b4a8-cc2bb171a614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:38.830941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933205-75e5-4335-b4a8-cc2bb171a614 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:38.831581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63933205-75e5-4335-b4a8-cc2bb171a614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:55:38.831881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63933205-75e5-4335-b4a8-cc2bb171a614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:38.833480 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63933205-75e5-4335-b4a8-cc2bb171a614 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:38.846253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0763e-a776-4344-85cf-306db7c5ede3 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:38.846755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1205/2439] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:55:38.847722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-76d0763e-a776-4344-85cf-306db7c5ede3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:38.857445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63933205-75e5-4335-b4a8-cc2bb171a614 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:38.867740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63933205-75e5-4335-b4a8-cc2bb171a614 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:38.873280 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c369f64-e7c0-47d2-8e5d-c3a6d099fe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:38.873923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c369f64-e7c0-47d2-8e5d-c3a6d099fe1c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:38.874375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c369f64-e7c0-47d2-8e5d-c3a6d099fe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:38.874642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c369f64-e7c0-47d2-8e5d-c3a6d099fe1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:38.875357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c369f64-e7c0-47d2-8e5d-c3a6d099fe1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:38.877759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c369f64-e7c0-47d2-8e5d-c3a6d099fe1c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:38.912997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0763e-a776-4344-85cf-306db7c5ede3 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:38.917252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37998-4362-44dd-a3f0-4d228f324e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:38.918017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37998-4362-44dd-a3f0-4d228f324e24 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:38.918408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37998-4362-44dd-a3f0-4d228f324e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:38.918776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37998-4362-44dd-a3f0-4d228f324e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:38.920983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bcc37998-4362-44dd-a3f0-4d228f324e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:38.981257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc37998-4362-44dd-a3f0-4d228f324e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:38.982786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc37998-4362-44dd-a3f0-4d228f324e24 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:38.992006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1206/2440] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:55:38 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:55:38.992737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1235/2441] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:55:38.996654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c369f64-e7c0-47d2-8e5d-c3a6d099fe1c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:39.079143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c369f64-e7c0-47d2-8e5d-c3a6d099fe1c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:39.091626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2ec27-dc87-4684-abb0-a74e44585740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:39.091626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2ec27-dc87-4684-abb0-a74e44585740 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:39.091626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2ec27-dc87-4684-abb0-a74e44585740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:39.091626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2ec27-dc87-4684-abb0-a74e44585740 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:39.092296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2ec27-dc87-4684-abb0-a74e44585740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:39.092296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2ec27-dc87-4684-abb0-a74e44585740 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:39.190124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1207/2442] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:55:38 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:39.194305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37998-4362-44dd-a3f0-4d228f324e24 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:39.206030 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37998-4362-44dd-a3f0-4d228f324e24 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:39.211631 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f41388-5474-4038-89c8-99d3dd4eb04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:39.212298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f41388-5474-4038-89c8-99d3dd4eb04b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:39.212576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f41388-5474-4038-89c8-99d3dd4eb04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:39.212975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f41388-5474-4038-89c8-99d3dd4eb04b admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:39.214218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f41388-5474-4038-89c8-99d3dd4eb04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:39.215258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f41388-5474-4038-89c8-99d3dd4eb04b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:39.229322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1236/2443] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:39.233233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2ec27-dc87-4684-abb0-a74e44585740 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:39.244614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2ec27-dc87-4684-abb0-a74e44585740 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:39.249988 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661e69a-5f4b-4e14-b8b2-7848255bc3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:39.251647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661e69a-5f4b-4e14-b8b2-7848255bc3b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:39.251647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661e69a-5f4b-4e14-b8b2-7848255bc3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:39.251647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661e69a-5f4b-4e14-b8b2-7848255bc3b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:39.252536 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661e69a-5f4b-4e14-b8b2-7848255bc3b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:39.254365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661e69a-5f4b-4e14-b8b2-7848255bc3b6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:39.272279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1208/2444] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:39.274240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f41388-5474-4038-89c8-99d3dd4eb04b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:39.287914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f41388-5474-4038-89c8-99d3dd4eb04b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:39.292741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2600f-5a35-43d8-ba57-d0de5acecd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:39.293405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2600f-5a35-43d8-ba57-d0de5acecd91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:39.293771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2600f-5a35-43d8-ba57-d0de5acecd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:39.294116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2600f-5a35-43d8-ba57-d0de5acecd91 admin admin] PATH_INFO: `/v3/projects/6bedacf1dd4e4398ae3acefcd58a93a8/users/db26f48da0af4723afb2c0a4c5f52d07/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:39.313420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2600f-5a35-43d8-ba57-d0de5acecd91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bedacf1dd4e4398ae3acefcd58a93a8, user_id=db26f48da0af4723afb2c0a4c5f52d07, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:39.316220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2600f-5a35-43d8-ba57-d0de5acecd91 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:39.396023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1237/2445] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:39 2022] PUT /identity/v3/projects/6bedacf1dd4e4398ae3acefcd58a93a8/users/db26f48da0af4723afb2c0a4c5f52d07/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:39.402716 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba547f33-fd57-4a1a-9c95-568681983518 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:39.403001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba547f33-fd57-4a1a-9c95-568681983518 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:39.403305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba547f33-fd57-4a1a-9c95-568681983518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:39.423668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ba547f33-fd57-4a1a-9c95-568681983518 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:39.424872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ba547f33-fd57-4a1a-9c95-568681983518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:39.437995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:39.489919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1209/2446] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:39.500216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2600f-5a35-43d8-ba57-d0de5acecd91 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:39.510951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:39.568311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2600f-5a35-43d8-ba57-d0de5acecd91 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:39.572286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807ffcb-0aa5-44d1-916e-d5d766f5c8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:39.572948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807ffcb-0aa5-44d1-916e-d5d766f5c8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:39.573273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807ffcb-0aa5-44d1-916e-d5d766f5c8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:39.573667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807ffcb-0aa5-44d1-916e-d5d766f5c8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:39.589286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807ffcb-0aa5-44d1-916e-d5d766f5c8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:39.590804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807ffcb-0aa5-44d1-916e-d5d766f5c8a2 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:39.601829 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1238/2447] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:46.196742 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477dbe7-d6cb-4429-8637-1e0c0a9a274b None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:46.197306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477dbe7-d6cb-4429-8637-1e0c0a9a274b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:46.197488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477dbe7-d6cb-4429-8637-1e0c0a9a274b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:46.235050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-5477dbe7-d6cb-4429-8637-1e0c0a9a274b None None] MFA Rules not processed for user `db26f48da0af4723afb2c0a4c5f52d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:46.236622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5477dbe7-d6cb-4429-8637-1e0c0a9a274b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:46.315305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1210/2448] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:46.321220 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807ffcb-0aa5-44d1-916e-d5d766f5c8a2 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:46.332652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807ffcb-0aa5-44d1-916e-d5d766f5c8a2 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:46.337757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ee259-6789-494d-9b62-1c19f67eb786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:46.338470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ee259-6789-494d-9b62-1c19f67eb786 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:46.338817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ee259-6789-494d-9b62-1c19f67eb786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:46.339307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ee259-6789-494d-9b62-1c19f67eb786 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:46.340103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ee259-6789-494d-9b62-1c19f67eb786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:46.342075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ee259-6789-494d-9b62-1c19f67eb786 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:46.453710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1239/2449] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:55:46 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:46.458129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477dbe7-d6cb-4429-8637-1e0c0a9a274b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:46.470776 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477dbe7-d6cb-4429-8637-1e0c0a9a274b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:46.534136 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732945d-ea11-40d4-adae-1eb25e81b83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:46.534746 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732945d-ea11-40d4-adae-1eb25e81b83a admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:46.535027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732945d-ea11-40d4-adae-1eb25e81b83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:46.535299 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732945d-ea11-40d4-adae-1eb25e81b83a admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:46.535927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732945d-ea11-40d4-adae-1eb25e81b83a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:46.537954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732945d-ea11-40d4-adae-1eb25e81b83a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:46.604662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1211/2450] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:55:46 2022] POST /identity/v3/users => generated 299 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:46.606044 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ee259-6789-494d-9b62-1c19f67eb786 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:46.617491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ee259-6789-494d-9b62-1c19f67eb786 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:46.623756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f1ef5-5dca-46a3-b84a-16c0ce73441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:46.624569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f1ef5-5dca-46a3-b84a-16c0ce73441b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:46.624848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f1ef5-5dca-46a3-b84a-16c0ce73441b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:46.625121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f1ef5-5dca-46a3-b84a-16c0ce73441b admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:46.625800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f1ef5-5dca-46a3-b84a-16c0ce73441b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:46.628287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f1ef5-5dca-46a3-b84a-16c0ce73441b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:46.647042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1240/2451] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:46.648442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732945d-ea11-40d4-adae-1eb25e81b83a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:46.661685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732945d-ea11-40d4-adae-1eb25e81b83a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:46.668036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254066fe-2a86-47d5-afd5-eb1732554713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:46.668683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254066fe-2a86-47d5-afd5-eb1732554713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:46.668972 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254066fe-2a86-47d5-afd5-eb1732554713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:46.669415 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254066fe-2a86-47d5-afd5-eb1732554713 admin admin] PATH_INFO: `/v3/projects/76bde2e6dfc040378bbc58faca557604/users/4cd1e14552324128b60c2d188a33f8c8/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:46.687545 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254066fe-2a86-47d5-afd5-eb1732554713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76bde2e6dfc040378bbc58faca557604, user_id=4cd1e14552324128b60c2d188a33f8c8, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:46.690311 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254066fe-2a86-47d5-afd5-eb1732554713 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:46.771992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1212/2452] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:46 2022] PUT /identity/v3/projects/76bde2e6dfc040378bbc58faca557604/users/4cd1e14552324128b60c2d188a33f8c8/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:46.778942 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f1ef5-5dca-46a3-b84a-16c0ce73441b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:46.796700 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f1ef5-5dca-46a3-b84a-16c0ce73441b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:46.866739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464af4c-4e70-43b9-93e4-fbe2d195c301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:46.867294 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464af4c-4e70-43b9-93e4-fbe2d195c301 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:46.867762 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464af4c-4e70-43b9-93e4-fbe2d195c301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:46.868045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464af4c-4e70-43b9-93e4-fbe2d195c301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:46.868921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464af4c-4e70-43b9-93e4-fbe2d195c301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:46.870644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464af4c-4e70-43b9-93e4-fbe2d195c301 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:46.888197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1241/2453] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:46.892332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254066fe-2a86-47d5-afd5-eb1732554713 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:46.911888 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254066fe-2a86-47d5-afd5-eb1732554713 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:46.919813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c31b74-7f29-4ba6-9f38-91661193859e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:46.920507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c31b74-7f29-4ba6-9f38-91661193859e admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:46.920776 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c31b74-7f29-4ba6-9f38-91661193859e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:46.921049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c31b74-7f29-4ba6-9f38-91661193859e admin admin] PATH_INFO: `/v3/projects/76bde2e6dfc040378bbc58faca557604/users/4cd1e14552324128b60c2d188a33f8c8/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:46.925701 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c31b74-7f29-4ba6-9f38-91661193859e admin admin] RBAC: Authorizing `identity:create_grant(project_id=76bde2e6dfc040378bbc58faca557604, user_id=4cd1e14552324128b60c2d188a33f8c8, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:46.929041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c31b74-7f29-4ba6-9f38-91661193859e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:46.938458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:46.944517 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1213/2454] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:46 2022] PUT /identity/v3/projects/76bde2e6dfc040378bbc58faca557604/users/4cd1e14552324128b60c2d188a33f8c8/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:47.114778 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464af4c-4e70-43b9-93e4-fbe2d195c301 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:47.116951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2464af4c-4e70-43b9-93e4-fbe2d195c301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:47.134483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464af4c-4e70-43b9-93e4-fbe2d195c301 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:47.202871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:47.203528 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:47.203985 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:47.204851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] PATH_INFO: `/v3/users/95e4f0998d5d4fb19e0f5f1d7ca0e69c` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:47.206267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] RBAC: Authorizing `identity:delete_user(user_id=95e4f0998d5d4fb19e0f5f1d7ca0e69c)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:47.208377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:47.226506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:47.421810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95e4f0998d5d4fb19e0f5f1d7ca0e69c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:47.521735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95e4f0998d5d4fb19e0f5f1d7ca0e69c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:47.523016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95e4f0998d5d4fb19e0f5f1d7ca0e69c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:47.531583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95e4f0998d5d4fb19e0f5f1d7ca0e69c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:47.544383 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95e4f0998d5d4fb19e0f5f1d7ca0e69c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:47.550710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1242/2455] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:55:47 2022] DELETE /identity/v3/users/95e4f0998d5d4fb19e0f5f1d7ca0e69c => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:47.555755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1adf2-7555-41d6-b93a-b165ed04bab1 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:47.564613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1adf2-7555-41d6-b93a-b165ed04bab1 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:47.564613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1adf2-7555-41d6-b93a-b165ed04bab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:47.577711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-74e1adf2-7555-41d6-b93a-b165ed04bab1 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:47.579061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-74e1adf2-7555-41d6-b93a-b165ed04bab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:47.653031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1214/2456] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:47.662927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:47.664370 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:47.730193 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26926711-c3cd-44a5-b292-500dc097818d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:47.734042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0011319-594d-4f5a-b718-84a08d78ec97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:47.735754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0011319-594d-4f5a-b718-84a08d78ec97 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:47.735754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0011319-594d-4f5a-b718-84a08d78ec97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:47.735754 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0011319-594d-4f5a-b718-84a08d78ec97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:47.750128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0011319-594d-4f5a-b718-84a08d78ec97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:47.752956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0011319-594d-4f5a-b718-84a08d78ec97 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:47.772962 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1243/2457] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:48.325566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1adf2-7555-41d6-b93a-b165ed04bab1 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:48.335684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1adf2-7555-41d6-b93a-b165ed04bab1 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:48.340348 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7cbbb-d97c-4291-938f-6042a8a2f27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:48.341070 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7cbbb-d97c-4291-938f-6042a8a2f27d service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:48.341418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7cbbb-d97c-4291-938f-6042a8a2f27d service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:48.341794 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7cbbb-d97c-4291-938f-6042a8a2f27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:48.353888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0011319-594d-4f5a-b718-84a08d78ec97 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:48.355613 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-10d7cbbb-d97c-4291-938f-6042a8a2f27d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:48.355761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c0011319-594d-4f5a-b718-84a08d78ec97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:48.379980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0011319-594d-4f5a-b718-84a08d78ec97 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:48.381489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7cbbb-d97c-4291-938f-6042a8a2f27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:48.383951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7cbbb-d97c-4291-938f-6042a8a2f27d service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:48.387827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:48.388610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:48.388995 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:48.389286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] PATH_INFO: `/v3/projects/d5d4703971b4479e8284c7bd7cbb028e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:48.391085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d4703971b4479e8284c7bd7cbb028e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:48.393758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:48.403389 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1215/2458] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:48.551170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d4703971b4479e8284c7bd7cbb028e', 'request_id': 'req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:48.575614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d4703971b4479e8284c7bd7cbb028e', 'request_id': 'req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:48.584872 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d4703971b4479e8284c7bd7cbb028e', 'request_id': 'req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:48.711154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] The token cache is being invalidate because project d5d4703971b4479e8284c7bd7cbb028e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:55:48.715929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:48.715929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:48.716161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1244/2459] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:55:48 2022] DELETE /identity/v3/projects/d5d4703971b4479e8284c7bd7cbb028e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:48.719998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7cbbb-d97c-4291-938f-6042a8a2f27d service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:48.736944 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7cbbb-d97c-4291-938f-6042a8a2f27d service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:48.853569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8e54d-d0b3-4b8c-ba32-2a98a6dcaed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:48.857218 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8e54d-d0b3-4b8c-ba32-2a98a6dcaed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:48.857862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8e54d-d0b3-4b8c-ba32-2a98a6dcaed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:48.858431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8e54d-d0b3-4b8c-ba32-2a98a6dcaed7 admin admin] PATH_INFO: `/v3/users/fa7f53bff12a42fe9b818617293c6981` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:48.861327 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8e54d-d0b3-4b8c-ba32-2a98a6dcaed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7f53bff12a42fe9b818617293c6981)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:48.864879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8e54d-d0b3-4b8c-ba32-2a98a6dcaed7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:49.151809 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8db8e54d-d0b3-4b8c-ba32-2a98a6dcaed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7f53bff12a42fe9b818617293c6981'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:49.234019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8db8e54d-d0b3-4b8c-ba32-2a98a6dcaed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7f53bff12a42fe9b818617293c6981'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:49.234019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8db8e54d-d0b3-4b8c-ba32-2a98a6dcaed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7f53bff12a42fe9b818617293c6981'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:49.243457 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8db8e54d-d0b3-4b8c-ba32-2a98a6dcaed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7f53bff12a42fe9b818617293c6981'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:49.275180 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-8db8e54d-d0b3-4b8c-ba32-2a98a6dcaed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7f53bff12a42fe9b818617293c6981'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:49.289557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1216/2460] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:55:48 2022] DELETE /identity/v3/users/fa7f53bff12a42fe9b818617293c6981 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:50.132582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:50.134209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:50.161417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4c3c8-ec98-4e22-ade3-2886aa9f368f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:50.180407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:50.298625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794f4ad-3207-432a-8865-efb07f892ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:50.299204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794f4ad-3207-432a-8865-efb07f892ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:50.299479 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794f4ad-3207-432a-8865-efb07f892ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:50.299751 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794f4ad-3207-432a-8865-efb07f892ee8 admin admin] PATH_INFO: `/v3/projects/9c29d1eff3fd484c810ba4c34128a3c3` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:50.300943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794f4ad-3207-432a-8865-efb07f892ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c29d1eff3fd484c810ba4c34128a3c3)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:50.304015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794f4ad-3207-432a-8865-efb07f892ee8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:50.427410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a794f4ad-3207-432a-8865-efb07f892ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c29d1eff3fd484c810ba4c34128a3c3', 'request_id': 'req-a794f4ad-3207-432a-8865-efb07f892ee8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:50.436872 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a794f4ad-3207-432a-8865-efb07f892ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c29d1eff3fd484c810ba4c34128a3c3', 'request_id': 'req-a794f4ad-3207-432a-8865-efb07f892ee8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:50.444709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a794f4ad-3207-432a-8865-efb07f892ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c29d1eff3fd484c810ba4c34128a3c3', 'request_id': 'req-a794f4ad-3207-432a-8865-efb07f892ee8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:50.527285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a794f4ad-3207-432a-8865-efb07f892ee8 admin admin] The token cache is being invalidate because project 9c29d1eff3fd484c810ba4c34128a3c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:55:50.527832 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a794f4ad-3207-432a-8865-efb07f892ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:50.528675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a794f4ad-3207-432a-8865-efb07f892ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:50.530500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1245/2461] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:55:50 2022] DELETE /identity/v3/projects/9c29d1eff3fd484c810ba4c34128a3c3 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:50.542079 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a821f23-f6c3-4c40-88dd-23b0d932c020 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:50.542657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a821f23-f6c3-4c40-88dd-23b0d932c020 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:50.542657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a821f23-f6c3-4c40-88dd-23b0d932c020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:50.565799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:50.573443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4a821f23-f6c3-4c40-88dd-23b0d932c020 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:50.575364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4a821f23-f6c3-4c40-88dd-23b0d932c020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:50.674082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1217/2462] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:50.682433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a75ddf-1dcc-409a-b150-5743f3dc93a7 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:50.682714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a75ddf-1dcc-409a-b150-5743f3dc93a7 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:50.683166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a75ddf-1dcc-409a-b150-5743f3dc93a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:50.708423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-26a75ddf-1dcc-409a-b150-5743f3dc93a7 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:50.709970 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-26a75ddf-1dcc-409a-b150-5743f3dc93a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:50.722388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1246/2463] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:50.726855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a821f23-f6c3-4c40-88dd-23b0d932c020 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:50.739634 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a821f23-f6c3-4c40-88dd-23b0d932c020 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:50.746947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c37416-585c-42e9-86e5-6ea077c8ebdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:50.750449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c37416-585c-42e9-86e5-6ea077c8ebdd admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:50.752290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c37416-585c-42e9-86e5-6ea077c8ebdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:50.752290 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c37416-585c-42e9-86e5-6ea077c8ebdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:50.752594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c37416-585c-42e9-86e5-6ea077c8ebdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:55:50.753014 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c37416-585c-42e9-86e5-6ea077c8ebdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:50.754859 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c37416-585c-42e9-86e5-6ea077c8ebdd admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:50.772834 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1218/2464] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:55:50.777216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a75ddf-1dcc-409a-b150-5743f3dc93a7 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:50.790552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a75ddf-1dcc-409a-b150-5743f3dc93a7 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:50.798504 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff58bb-7bcd-45b8-a613-60bef3414a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:50.799165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff58bb-7bcd-45b8-a613-60bef3414a8b admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:50.799442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff58bb-7bcd-45b8-a613-60bef3414a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:50.799697 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff58bb-7bcd-45b8-a613-60bef3414a8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:50.800347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff58bb-7bcd-45b8-a613-60bef3414a8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:50.802351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff58bb-7bcd-45b8-a613-60bef3414a8b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:50.932883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1247/2465] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:55:50 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:50.937367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c37416-585c-42e9-86e5-6ea077c8ebdd admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:50.953409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c37416-585c-42e9-86e5-6ea077c8ebdd admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:51.090421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476fe48-d7fe-4d3c-9f15-f0d15b50cf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:51.092567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476fe48-d7fe-4d3c-9f15-f0d15b50cf75 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:51.092567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476fe48-d7fe-4d3c-9f15-f0d15b50cf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:51.092567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476fe48-d7fe-4d3c-9f15-f0d15b50cf75 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:51.092567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476fe48-d7fe-4d3c-9f15-f0d15b50cf75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:51.099424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476fe48-d7fe-4d3c-9f15-f0d15b50cf75 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:51.195452 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1219/2466] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:55:50 2022] POST /identity/v3/users => generated 293 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:51.200255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff58bb-7bcd-45b8-a613-60bef3414a8b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:51.219132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff58bb-7bcd-45b8-a613-60bef3414a8b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:51.237363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b040c5a-3e58-423d-9bd0-ee3a5dcbe513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:51.238424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b040c5a-3e58-423d-9bd0-ee3a5dcbe513 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:51.238940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b040c5a-3e58-423d-9bd0-ee3a5dcbe513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:51.239428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b040c5a-3e58-423d-9bd0-ee3a5dcbe513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:51.240332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b040c5a-3e58-423d-9bd0-ee3a5dcbe513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:51.244637 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b040c5a-3e58-423d-9bd0-ee3a5dcbe513 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:51.272463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1248/2467] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:51.276869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476fe48-d7fe-4d3c-9f15-f0d15b50cf75 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:51.292810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476fe48-d7fe-4d3c-9f15-f0d15b50cf75 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:51.306301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f35277-f74f-4cad-90ac-797afb2384d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:51.307296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f35277-f74f-4cad-90ac-797afb2384d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:51.307426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f35277-f74f-4cad-90ac-797afb2384d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:51.307607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f35277-f74f-4cad-90ac-797afb2384d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:51.308430 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f35277-f74f-4cad-90ac-797afb2384d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:51.314302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f35277-f74f-4cad-90ac-797afb2384d8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:51.343461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1220/2468] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:51.346178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b040c5a-3e58-423d-9bd0-ee3a5dcbe513 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:51.362095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b040c5a-3e58-423d-9bd0-ee3a5dcbe513 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:51.374248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5ef4c-dc1d-493b-a994-da9d2792aa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:51.378308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5ef4c-dc1d-493b-a994-da9d2792aa18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:51.389652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5ef4c-dc1d-493b-a994-da9d2792aa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:51.390204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5ef4c-dc1d-493b-a994-da9d2792aa18 admin admin] PATH_INFO: `/v3/projects/02412b4726d44016bcb210890d68945d/users/553d9e6c816b44a1a58ea3c7b2cf9f1e/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:51.446029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5ef4c-dc1d-493b-a994-da9d2792aa18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02412b4726d44016bcb210890d68945d, user_id=553d9e6c816b44a1a58ea3c7b2cf9f1e, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:51.449615 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5ef4c-dc1d-493b-a994-da9d2792aa18 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:51.540709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1249/2469] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:51 2022] PUT /identity/v3/projects/02412b4726d44016bcb210890d68945d/users/553d9e6c816b44a1a58ea3c7b2cf9f1e/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:51.566817 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19297f89-1023-4f98-b832-0a87af892ff4 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:51.567688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19297f89-1023-4f98-b832-0a87af892ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:51.568151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19297f89-1023-4f98-b832-0a87af892ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:51.672795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-19297f89-1023-4f98-b832-0a87af892ff4 None None] MFA Rules not processed for user `553d9e6c816b44a1a58ea3c7b2cf9f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:51.678286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-19297f89-1023-4f98-b832-0a87af892ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:51.946049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1221/2470] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:51.947351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5ef4c-dc1d-493b-a994-da9d2792aa18 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:52.026409 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:52.047141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5ef4c-dc1d-493b-a994-da9d2792aa18 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:52.052860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475156c-a3ea-43fa-a4df-c751c0fcb873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:52.054151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475156c-a3ea-43fa-a4df-c751c0fcb873 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:52.055620 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475156c-a3ea-43fa-a4df-c751c0fcb873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:52.055949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475156c-a3ea-43fa-a4df-c751c0fcb873 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:52.056698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475156c-a3ea-43fa-a4df-c751c0fcb873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:52.058912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475156c-a3ea-43fa-a4df-c751c0fcb873 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:52.222254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1250/2471] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:55:51 2022] POST /identity/v3/projects => generated 387 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:52.223409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19297f89-1023-4f98-b832-0a87af892ff4 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:52.237644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:52.238707 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19297f89-1023-4f98-b832-0a87af892ff4 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:52.441265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ced71c-21ea-4eda-b212-e167d69f85e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:52.441265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ced71c-21ea-4eda-b212-e167d69f85e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:52.441265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ced71c-21ea-4eda-b212-e167d69f85e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:52.441265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ced71c-21ea-4eda-b212-e167d69f85e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:52.442395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ced71c-21ea-4eda-b212-e167d69f85e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:52.442395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ced71c-21ea-4eda-b212-e167d69f85e7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:52.508999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1222/2472] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:55:52 2022] POST /identity/v3/users => generated 300 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:52.513026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475156c-a3ea-43fa-a4df-c751c0fcb873 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:52.542707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475156c-a3ea-43fa-a4df-c751c0fcb873 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:52.552576 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3e95b-3f35-4872-8eff-cb8ff9ee2629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:52.553223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3e95b-3f35-4872-8eff-cb8ff9ee2629 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:52.553505 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3e95b-3f35-4872-8eff-cb8ff9ee2629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:52.553811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3e95b-3f35-4872-8eff-cb8ff9ee2629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:52.554474 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3e95b-3f35-4872-8eff-cb8ff9ee2629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:52.556405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3e95b-3f35-4872-8eff-cb8ff9ee2629 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:52.570441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1251/2473] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:52.574534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ced71c-21ea-4eda-b212-e167d69f85e7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:52.587987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ced71c-21ea-4eda-b212-e167d69f85e7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:52.594830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9eb13e-e5b3-4c0c-a275-fdc3bb1d8906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:52.595708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eb13e-e5b3-4c0c-a275-fdc3bb1d8906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:52.596130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eb13e-e5b3-4c0c-a275-fdc3bb1d8906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:52.596506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eb13e-e5b3-4c0c-a275-fdc3bb1d8906 admin admin] PATH_INFO: `/v3/projects/0dba4221101d449188c1ea482cf1b1c7/users/8402edcd06c747258f9dfa03bcfca243/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:52.622724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9eb13e-e5b3-4c0c-a275-fdc3bb1d8906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dba4221101d449188c1ea482cf1b1c7, user_id=8402edcd06c747258f9dfa03bcfca243, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:52.630782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9eb13e-e5b3-4c0c-a275-fdc3bb1d8906 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:52.742674 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1223/2474] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:52 2022] PUT /identity/v3/projects/0dba4221101d449188c1ea482cf1b1c7/users/8402edcd06c747258f9dfa03bcfca243/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:55:52.746870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3e95b-3f35-4872-8eff-cb8ff9ee2629 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:52.758993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3e95b-3f35-4872-8eff-cb8ff9ee2629 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:52.858145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863f43f-6aca-4f27-a8bb-6f6da49296ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:52.858145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863f43f-6aca-4f27-a8bb-6f6da49296ff admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:52.858145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863f43f-6aca-4f27-a8bb-6f6da49296ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:52.858145 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863f43f-6aca-4f27-a8bb-6f6da49296ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:52.858562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863f43f-6aca-4f27-a8bb-6f6da49296ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:52.860311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863f43f-6aca-4f27-a8bb-6f6da49296ff admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:52.877741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1252/2475] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:52.881994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9eb13e-e5b3-4c0c-a275-fdc3bb1d8906 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:52.896046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9eb13e-e5b3-4c0c-a275-fdc3bb1d8906 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:52.913091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3a44c-8f67-49d8-a57b-a17699ab081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:52.917780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3a44c-8f67-49d8-a57b-a17699ab081d admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:52.917780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3a44c-8f67-49d8-a57b-a17699ab081d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:52.917780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3a44c-8f67-49d8-a57b-a17699ab081d admin admin] PATH_INFO: `/v3/projects/0dba4221101d449188c1ea482cf1b1c7/users/8402edcd06c747258f9dfa03bcfca243/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:52.920400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3a44c-8f67-49d8-a57b-a17699ab081d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dba4221101d449188c1ea482cf1b1c7, user_id=8402edcd06c747258f9dfa03bcfca243, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:52.924777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3a44c-8f67-49d8-a57b-a17699ab081d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:52.964100 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1224/2476] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:52 2022] PUT /identity/v3/projects/0dba4221101d449188c1ea482cf1b1c7/users/8402edcd06c747258f9dfa03bcfca243/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:52.965996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577a770-6a71-4044-af61-1d0bdcae86db None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:52.966388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577a770-6a71-4044-af61-1d0bdcae86db None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:52.969939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577a770-6a71-4044-af61-1d0bdcae86db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:53.052920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e577a770-6a71-4044-af61-1d0bdcae86db None None] MFA Rules not processed for user `8402edcd06c747258f9dfa03bcfca243`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:53.054732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e577a770-6a71-4044-af61-1d0bdcae86db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:53.163613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1253/2477] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:53.187210 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3a44c-8f67-49d8-a57b-a17699ab081d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:53.189971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9cb3a44c-8f67-49d8-a57b-a17699ab081d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:53.315806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3a44c-8f67-49d8-a57b-a17699ab081d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:53.319945 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13a546-90d1-4809-860e-a409f15be592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:53.320644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13a546-90d1-4809-860e-a409f15be592 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:53.320925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13a546-90d1-4809-860e-a409f15be592 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:53.324968 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13a546-90d1-4809-860e-a409f15be592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:53.332566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13a546-90d1-4809-860e-a409f15be592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:53.335392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13a546-90d1-4809-860e-a409f15be592 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:53.355167 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1225/2478] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:53 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:53.404570 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577a770-6a71-4044-af61-1d0bdcae86db None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:53.494603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:53.530516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577a770-6a71-4044-af61-1d0bdcae86db None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:53.542351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:53.544786 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:53.545885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:53.546356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] PATH_INFO: `/v3/users/553d9e6c816b44a1a58ea3c7b2cf9f1e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:53.553298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=553d9e6c816b44a1a58ea3c7b2cf9f1e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:53.555826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:53.862651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '553d9e6c816b44a1a58ea3c7b2cf9f1e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:53.943473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '553d9e6c816b44a1a58ea3c7b2cf9f1e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:53.944217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '553d9e6c816b44a1a58ea3c7b2cf9f1e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:53.953359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '553d9e6c816b44a1a58ea3c7b2cf9f1e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:53.974200 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '553d9e6c816b44a1a58ea3c7b2cf9f1e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:53.989012 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1254/2479] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:55:53 2022] DELETE /identity/v3/users/553d9e6c816b44a1a58ea3c7b2cf9f1e => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:53.990092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208cb5c-4548-4ebb-b469-7b9ee9abd12c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:53.990921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208cb5c-4548-4ebb-b469-7b9ee9abd12c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:53.991364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208cb5c-4548-4ebb-b469-7b9ee9abd12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:54.025926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e208cb5c-4548-4ebb-b469-7b9ee9abd12c None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:54.027111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e208cb5c-4548-4ebb-b469-7b9ee9abd12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:54.152219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:54.167761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1226/2480] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:55:54.191563 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:54.193129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:54.295322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5272d1-fb50-4608-bc3f-9c486a45ff6a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:54.303472 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e26fc7-0dad-4133-808a-566178d5e8f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:54.304452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e26fc7-0dad-4133-808a-566178d5e8f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:54.304739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e26fc7-0dad-4133-808a-566178d5e8f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:54.306310 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e26fc7-0dad-4133-808a-566178d5e8f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:54.326494 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e26fc7-0dad-4133-808a-566178d5e8f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:54.328279 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e26fc7-0dad-4133-808a-566178d5e8f9 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:54.364784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1255/2481] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:54.594787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208cb5c-4548-4ebb-b469-7b9ee9abd12c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:54.596749 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e208cb5c-4548-4ebb-b469-7b9ee9abd12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:54.776890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208cb5c-4548-4ebb-b469-7b9ee9abd12c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:54.783098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae9acb-f3c6-4094-9711-6061e44f3552 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0aa6b215f646c280af245a89ccf849', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:54.783840 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae9acb-f3c6-4094-9711-6061e44f3552 service placement] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:54.784675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae9acb-f3c6-4094-9711-6061e44f3552 service placement] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:54.784675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae9acb-f3c6-4094-9711-6061e44f3552 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:54.786642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-47ae9acb-f3c6-4094-9711-6061e44f3552 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:54.804666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae9acb-f3c6-4094-9711-6061e44f3552 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:54.807397 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae9acb-f3c6-4094-9711-6061e44f3552 service placement] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:54.821787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1227/2482] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:55.340678 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e26fc7-0dad-4133-808a-566178d5e8f9 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:55.342771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-77e26fc7-0dad-4133-808a-566178d5e8f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:55.369162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e26fc7-0dad-4133-808a-566178d5e8f9 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:55.379132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:55.380132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:55.380647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:55.381103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] PATH_INFO: `/v3/projects/02412b4726d44016bcb210890d68945d` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:55.384002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02412b4726d44016bcb210890d68945d)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:55.386639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:55.531982 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02412b4726d44016bcb210890d68945d', 'request_id': 'req-b099bfd6-2d17-4d24-880a-f4ca237e85a7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:55.544841 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:55.557316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d72405-dccb-4335-81af-3bc4e698e87a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:55.560970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d72405-dccb-4335-81af-3bc4e698e87a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:55.561317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d72405-dccb-4335-81af-3bc4e698e87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:55.565454 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02412b4726d44016bcb210890d68945d', 'request_id': 'req-b099bfd6-2d17-4d24-880a-f4ca237e85a7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:55.611772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02412b4726d44016bcb210890d68945d', 'request_id': 'req-b099bfd6-2d17-4d24-880a-f4ca237e85a7'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:55.647657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-65d72405-dccb-4335-81af-3bc4e698e87a None None] MFA Rules not processed for user `4cd1e14552324128b60c2d188a33f8c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:55.649929 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-65d72405-dccb-4335-81af-3bc4e698e87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:55.688026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] The token cache is being invalidate because project 02412b4726d44016bcb210890d68945d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:55:55.688319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:55.691775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:55.697486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1256/2483] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:55:55 2022] DELETE /identity/v3/projects/02412b4726d44016bcb210890d68945d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:55.702290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:55.706759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:55.828386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099bfd6-2d17-4d24-880a-f4ca237e85a7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:55.837442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86d7ed-95b6-4426-ad34-d5d49f6e5063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:55.838258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86d7ed-95b6-4426-ad34-d5d49f6e5063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:55.838717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86d7ed-95b6-4426-ad34-d5d49f6e5063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:55.839080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86d7ed-95b6-4426-ad34-d5d49f6e5063 admin admin] PATH_INFO: `/v3/users/8402edcd06c747258f9dfa03bcfca243` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:55.840380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86d7ed-95b6-4426-ad34-d5d49f6e5063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8402edcd06c747258f9dfa03bcfca243)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:55.843118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86d7ed-95b6-4426-ad34-d5d49f6e5063 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:55.857894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1228/2484] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:55.872176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d72405-dccb-4335-81af-3bc4e698e87a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:55.873506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-65d72405-dccb-4335-81af-3bc4e698e87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:56.062448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cb86d7ed-95b6-4426-ad34-d5d49f6e5063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8402edcd06c747258f9dfa03bcfca243'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:56.112221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d72405-dccb-4335-81af-3bc4e698e87a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:56.118334 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c1d10-5808-4de4-a903-85e00c1dc993 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a58f9101f2d4785b96f0cc55be4330b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:56.118966 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c1d10-5808-4de4-a903-85e00c1dc993 service glance] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:56.119438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c1d10-5808-4de4-a903-85e00c1dc993 service glance] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:56.122233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c1d10-5808-4de4-a903-85e00c1dc993 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:56.122233 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d50c1d10-5808-4de4-a903-85e00c1dc993 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:56.138501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cb86d7ed-95b6-4426-ad34-d5d49f6e5063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8402edcd06c747258f9dfa03bcfca243'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:56.139445 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cb86d7ed-95b6-4426-ad34-d5d49f6e5063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8402edcd06c747258f9dfa03bcfca243'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:56.154780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cb86d7ed-95b6-4426-ad34-d5d49f6e5063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8402edcd06c747258f9dfa03bcfca243'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:56.172710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-cb86d7ed-95b6-4426-ad34-d5d49f6e5063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8402edcd06c747258f9dfa03bcfca243'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:56.180462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1257/2485] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:55:55 2022] DELETE /identity/v3/users/8402edcd06c747258f9dfa03bcfca243 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:56.218052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:56.341769 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c1d10-5808-4de4-a903-85e00c1dc993 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:56.343568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c1d10-5808-4de4-a903-85e00c1dc993 service glance] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:56.356291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1229/2486] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:55:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:55:56.692149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212084ec-4d45-4ccd-9dff-a2be2b89cf5e None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:56.693113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212084ec-4d45-4ccd-9dff-a2be2b89cf5e None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:56.693688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212084ec-4d45-4ccd-9dff-a2be2b89cf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:56.748873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-212084ec-4d45-4ccd-9dff-a2be2b89cf5e None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:56.752263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-212084ec-4d45-4ccd-9dff-a2be2b89cf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:56.852715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1258/2487] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:55:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:56.887338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c1d10-5808-4de4-a903-85e00c1dc993 service glance] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:56.889366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d50c1d10-5808-4de4-a903-85e00c1dc993 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:57.051584 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c1d10-5808-4de4-a903-85e00c1dc993 service glance] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:57.118465 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212084ec-4d45-4ccd-9dff-a2be2b89cf5e None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:57.120244 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-212084ec-4d45-4ccd-9dff-a2be2b89cf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:57.135250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:57.194560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b3701-d5b6-44c3-b573-2e656920659f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:57.196111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b3701-d5b6-44c3-b573-2e656920659f service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:57.196111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b3701-d5b6-44c3-b573-2e656920659f service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:57.196765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b3701-d5b6-44c3-b573-2e656920659f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:57.243041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b3701-d5b6-44c3-b573-2e656920659f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:57.247519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b3701-d5b6-44c3-b573-2e656920659f service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:57.271922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1230/2488] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:55:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:55:57.295991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212084ec-4d45-4ccd-9dff-a2be2b89cf5e None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:57.313114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9112c33-4ea0-4082-8788-7c7a78889f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:57.314526 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9112c33-4ea0-4082-8788-7c7a78889f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:57.315087 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9112c33-4ea0-4082-8788-7c7a78889f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:57.317255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9112c33-4ea0-4082-8788-7c7a78889f2d admin admin] PATH_INFO: `/v3/projects/0dba4221101d449188c1ea482cf1b1c7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:57.318554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9112c33-4ea0-4082-8788-7c7a78889f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dba4221101d449188c1ea482cf1b1c7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:57.323135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9112c33-4ea0-4082-8788-7c7a78889f2d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:57.472393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e9112c33-4ea0-4082-8788-7c7a78889f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dba4221101d449188c1ea482cf1b1c7', 'request_id': 'req-e9112c33-4ea0-4082-8788-7c7a78889f2d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:57.497948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e9112c33-4ea0-4082-8788-7c7a78889f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dba4221101d449188c1ea482cf1b1c7', 'request_id': 'req-e9112c33-4ea0-4082-8788-7c7a78889f2d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:57.523313 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e9112c33-4ea0-4082-8788-7c7a78889f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dba4221101d449188c1ea482cf1b1c7', 'request_id': 'req-e9112c33-4ea0-4082-8788-7c7a78889f2d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:57.610253 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e9112c33-4ea0-4082-8788-7c7a78889f2d admin admin] The token cache is being invalidate because project 0dba4221101d449188c1ea482cf1b1c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:55:57.610572 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e9112c33-4ea0-4082-8788-7c7a78889f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:57.611269 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-e9112c33-4ea0-4082-8788-7c7a78889f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:55:57.613083 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1259/2489] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:55:57 2022] DELETE /identity/v3/projects/0dba4221101d449188c1ea482cf1b1c7 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:55:57.618149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd45a42-bc0e-49f7-a92e-0125b4259287 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:57.618453 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd45a42-bc0e-49f7-a92e-0125b4259287 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:57.618724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd45a42-bc0e-49f7-a92e-0125b4259287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:57.674800 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4cd45a42-bc0e-49f7-a92e-0125b4259287 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:57.676275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4cd45a42-bc0e-49f7-a92e-0125b4259287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:57.852107 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:57.880212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1231/2490] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:57.882932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1423604-63ee-407c-899f-6e48bbc3c40a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:57.883458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1423604-63ee-407c-899f-6e48bbc3c40a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:57.883555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1423604-63ee-407c-899f-6e48bbc3c40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:57.946377 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d1423604-63ee-407c-899f-6e48bbc3c40a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:57.949462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d1423604-63ee-407c-899f-6e48bbc3c40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:57.963154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1260/2491] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:57.968556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd45a42-bc0e-49f7-a92e-0125b4259287 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:57.983105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd45a42-bc0e-49f7-a92e-0125b4259287 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:57.989304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc74bd-2690-404b-bcba-2b1845c4815f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:57.990015 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc74bd-2690-404b-bcba-2b1845c4815f admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:57.990539 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc74bd-2690-404b-bcba-2b1845c4815f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:57.990860 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc74bd-2690-404b-bcba-2b1845c4815f admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:57.994947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc74bd-2690-404b-bcba-2b1845c4815f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:55:57.994947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc74bd-2690-404b-bcba-2b1845c4815f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:57.994947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc74bd-2690-404b-bcba-2b1845c4815f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:58.023069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1232/2492] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:55:58.028012 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1423604-63ee-407c-899f-6e48bbc3c40a None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:58.042494 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1423604-63ee-407c-899f-6e48bbc3c40a None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:58.051155 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d29529-9924-425d-a26a-e074691fdb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:58.051922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d29529-9924-425d-a26a-e074691fdb83 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:58.052408 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d29529-9924-425d-a26a-e074691fdb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:58.052920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d29529-9924-425d-a26a-e074691fdb83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:58.053768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d29529-9924-425d-a26a-e074691fdb83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:58.056023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d29529-9924-425d-a26a-e074691fdb83 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:58.153544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1261/2493] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:55:58 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:58.159997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc74bd-2690-404b-bcba-2b1845c4815f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:58.172077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc74bd-2690-404b-bcba-2b1845c4815f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:58.275806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95282887-56d1-49cd-87ad-7f5667016d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:58.276464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95282887-56d1-49cd-87ad-7f5667016d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:58.276745 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95282887-56d1-49cd-87ad-7f5667016d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:58.277022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95282887-56d1-49cd-87ad-7f5667016d74 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:58.277700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95282887-56d1-49cd-87ad-7f5667016d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:58.279924 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95282887-56d1-49cd-87ad-7f5667016d74 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:58.367977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1233/2494] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:55:58 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:58.373042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d29529-9924-425d-a26a-e074691fdb83 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:58.397685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d29529-9924-425d-a26a-e074691fdb83 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:58.410268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485824a-dedb-465b-8c18-9a25b1720dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:58.413154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485824a-dedb-465b-8c18-9a25b1720dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:58.413511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485824a-dedb-465b-8c18-9a25b1720dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:58.413887 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485824a-dedb-465b-8c18-9a25b1720dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:58.419283 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485824a-dedb-465b-8c18-9a25b1720dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:58.421264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485824a-dedb-465b-8c18-9a25b1720dc3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:58.443846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1262/2495] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:58.447644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95282887-56d1-49cd-87ad-7f5667016d74 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:58.462474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95282887-56d1-49cd-87ad-7f5667016d74 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:58.477083 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2e423-c8f6-4988-8e41-e99bc1ffd694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:58.477842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2e423-c8f6-4988-8e41-e99bc1ffd694 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:58.478205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2e423-c8f6-4988-8e41-e99bc1ffd694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:58.478552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2e423-c8f6-4988-8e41-e99bc1ffd694 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:58.479437 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2e423-c8f6-4988-8e41-e99bc1ffd694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:58.481126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2e423-c8f6-4988-8e41-e99bc1ffd694 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:58.506996 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1234/2496] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:58.511758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485824a-dedb-465b-8c18-9a25b1720dc3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:58.527419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485824a-dedb-465b-8c18-9a25b1720dc3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:58.539507 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f6fbf-d3c0-492a-be00-4c4be9a07224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:58.542956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6fbf-d3c0-492a-be00-4c4be9a07224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:58.543312 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6fbf-d3c0-492a-be00-4c4be9a07224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:58.543659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6fbf-d3c0-492a-be00-4c4be9a07224 admin admin] PATH_INFO: `/v3/projects/59233f3b59524631888657bc9f402e12/users/59dc7e2496a24e57aa923173369b5d99/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:58.572483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f6fbf-d3c0-492a-be00-4c4be9a07224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59233f3b59524631888657bc9f402e12, user_id=59dc7e2496a24e57aa923173369b5d99, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:58.580144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f6fbf-d3c0-492a-be00-4c4be9a07224 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:58.664935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1263/2497] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:58 2022] PUT /identity/v3/projects/59233f3b59524631888657bc9f402e12/users/59dc7e2496a24e57aa923173369b5d99/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:58.678041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf63118-4596-48ae-b120-03514c8cc4e0 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:58.678653 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf63118-4596-48ae-b120-03514c8cc4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:58.679146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf63118-4596-48ae-b120-03514c8cc4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:58.741919 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4cf63118-4596-48ae-b120-03514c8cc4e0 None None] MFA Rules not processed for user `59dc7e2496a24e57aa923173369b5d99`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:55:58.743414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4cf63118-4596-48ae-b120-03514c8cc4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:55:58.934367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1235/2498] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:55:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:55:58.939863 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f6fbf-d3c0-492a-be00-4c4be9a07224 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:59.035460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:59.071150 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f6fbf-d3c0-492a-be00-4c4be9a07224 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:59.081235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9598611-aac0-484a-be0c-ebb71a50cc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:59.082653 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9598611-aac0-484a-be0c-ebb71a50cc35 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:59.083681 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9598611-aac0-484a-be0c-ebb71a50cc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:59.084198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9598611-aac0-484a-be0c-ebb71a50cc35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:59.085108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9598611-aac0-484a-be0c-ebb71a50cc35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:59.087719 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9598611-aac0-484a-be0c-ebb71a50cc35 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:59.269160 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1264/2499] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:55:58 2022] POST /identity/v3/projects => generated 385 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:55:59.277861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf63118-4596-48ae-b120-03514c8cc4e0 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:59.304947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf63118-4596-48ae-b120-03514c8cc4e0 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:59.407909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:55:59.416032 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82a914-80c2-450c-82bc-e4e7033fc718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:59.417689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82a914-80c2-450c-82bc-e4e7033fc718 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:59.418126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82a914-80c2-450c-82bc-e4e7033fc718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:59.418741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82a914-80c2-450c-82bc-e4e7033fc718 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:59.419564 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82a914-80c2-450c-82bc-e4e7033fc718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:59.421897 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82a914-80c2-450c-82bc-e4e7033fc718 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:59.528569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1236/2500] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:55:59 2022] POST /identity/v3/users => generated 299 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:55:59.535165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9598611-aac0-484a-be0c-ebb71a50cc35 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:59.551168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9598611-aac0-484a-be0c-ebb71a50cc35 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:59.558457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ef793-1d8c-481b-a991-8f49eb683c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:59.559371 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ef793-1d8c-481b-a991-8f49eb683c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:59.559932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ef793-1d8c-481b-a991-8f49eb683c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:59.560254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ef793-1d8c-481b-a991-8f49eb683c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:59.560926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ef793-1d8c-481b-a991-8f49eb683c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:59.562787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ef793-1d8c-481b-a991-8f49eb683c7c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:59.581500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82a914-80c2-450c-82bc-e4e7033fc718 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:59.582346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1265/2501] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:55:59.596338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82a914-80c2-450c-82bc-e4e7033fc718 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:59.635520 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b2fee-2904-4f3d-ab80-175202c45963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:59.642388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b2fee-2904-4f3d-ab80-175202c45963 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:59.642388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b2fee-2904-4f3d-ab80-175202c45963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:59.644606 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b2fee-2904-4f3d-ab80-175202c45963 admin admin] PATH_INFO: `/v3/projects/569b9892afde4f89b6573fd62183d929/users/84d30adb5a4d4824b4db3a8884f59b69/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:59.698912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b2fee-2904-4f3d-ab80-175202c45963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=569b9892afde4f89b6573fd62183d929, user_id=84d30adb5a4d4824b4db3a8884f59b69, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:59.704644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b2fee-2904-4f3d-ab80-175202c45963 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:59.798938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1237/2502] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:55:59 2022] PUT /identity/v3/projects/569b9892afde4f89b6573fd62183d929/users/84d30adb5a4d4824b4db3a8884f59b69/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:55:59.803355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ef793-1d8c-481b-a991-8f49eb683c7c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:55:59.825508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ef793-1d8c-481b-a991-8f49eb683c7c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:55:59.969846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10b4bc-75fc-4aaf-bd6c-2f377c0af44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:55:59.970884 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b4bc-75fc-4aaf-bd6c-2f377c0af44e admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:55:59.971311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b4bc-75fc-4aaf-bd6c-2f377c0af44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:55:59.971753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b4bc-75fc-4aaf-bd6c-2f377c0af44e admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:55:59.972851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10b4bc-75fc-4aaf-bd6c-2f377c0af44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:55:59.975742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10b4bc-75fc-4aaf-bd6c-2f377c0af44e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:55:59.998703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1266/2503] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:00.005807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b2fee-2904-4f3d-ab80-175202c45963 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:00.032299 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b2fee-2904-4f3d-ab80-175202c45963 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:00.044461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e420c-f180-4686-9293-b9d77e0c8739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:00.045514 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e420c-f180-4686-9293-b9d77e0c8739 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:00.046309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e420c-f180-4686-9293-b9d77e0c8739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:00.046796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e420c-f180-4686-9293-b9d77e0c8739 admin admin] PATH_INFO: `/v3/projects/569b9892afde4f89b6573fd62183d929/users/84d30adb5a4d4824b4db3a8884f59b69/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:00.052942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e420c-f180-4686-9293-b9d77e0c8739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=569b9892afde4f89b6573fd62183d929, user_id=84d30adb5a4d4824b4db3a8884f59b69, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:00.056935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e420c-f180-4686-9293-b9d77e0c8739 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:00.075931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1238/2504] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:00 2022] PUT /identity/v3/projects/569b9892afde4f89b6573fd62183d929/users/84d30adb5a4d4824b4db3a8884f59b69/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:00.088162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423712b6-9aa6-4e06-9381-e46d133dbeb8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:00.089123 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423712b6-9aa6-4e06-9381-e46d133dbeb8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:00.089645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423712b6-9aa6-4e06-9381-e46d133dbeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:00.137226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-423712b6-9aa6-4e06-9381-e46d133dbeb8 None None] MFA Rules not processed for user `84d30adb5a4d4824b4db3a8884f59b69`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:00.138506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-423712b6-9aa6-4e06-9381-e46d133dbeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:00.367158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1267/2505] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:00.378286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e420c-f180-4686-9293-b9d77e0c8739 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:00.382799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-8c8e420c-f180-4686-9293-b9d77e0c8739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:00.579968 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e420c-f180-4686-9293-b9d77e0c8739 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:00.598782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503f578-7ecd-4c89-96ce-4dc2e5c520b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:00.599668 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503f578-7ecd-4c89-96ce-4dc2e5c520b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:00.599941 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503f578-7ecd-4c89-96ce-4dc2e5c520b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:00.600206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503f578-7ecd-4c89-96ce-4dc2e5c520b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:00.616297 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423712b6-9aa6-4e06-9381-e46d133dbeb8 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:00.617979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-423712b6-9aa6-4e06-9381-e46d133dbeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:00.624096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503f578-7ecd-4c89-96ce-4dc2e5c520b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:00.625714 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503f578-7ecd-4c89-96ce-4dc2e5c520b0 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:00.643918 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1239/2506] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:00.748591 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:00.812165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423712b6-9aa6-4e06-9381-e46d133dbeb8 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:00.818505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503f578-7ecd-4c89-96ce-4dc2e5c520b0 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:00.819537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2d618-eea5-4cb6-9981-85f5e0e0ca77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:00.820150 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2d618-eea5-4cb6-9981-85f5e0e0ca77 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:00.820458 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2d618-eea5-4cb6-9981-85f5e0e0ca77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:00.821240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2d618-eea5-4cb6-9981-85f5e0e0ca77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:00.822925 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1503f578-7ecd-4c89-96ce-4dc2e5c520b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:00.828081 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-93b2d618-eea5-4cb6-9981-85f5e0e0ca77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:00.970960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2d618-eea5-4cb6-9981-85f5e0e0ca77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:00.972847 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2d618-eea5-4cb6-9981-85f5e0e0ca77 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:00.986670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1268/2507] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:56:01.076645 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503f578-7ecd-4c89-96ce-4dc2e5c520b0 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:01.094075 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636ddf0-6f5b-49cf-8e14-9aeb3e4e4a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:01.095093 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636ddf0-6f5b-49cf-8e14-9aeb3e4e4a7a service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:01.095581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636ddf0-6f5b-49cf-8e14-9aeb3e4e4a7a service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:01.096008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636ddf0-6f5b-49cf-8e14-9aeb3e4e4a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:01.114762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c636ddf0-6f5b-49cf-8e14-9aeb3e4e4a7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:01.183647 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:01.213355 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636ddf0-6f5b-49cf-8e14-9aeb3e4e4a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:01.215035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636ddf0-6f5b-49cf-8e14-9aeb3e4e4a7a service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:01.246363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1240/2508] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:56:02.149444 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2d618-eea5-4cb6-9981-85f5e0e0ca77 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:02.151885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-93b2d618-eea5-4cb6-9981-85f5e0e0ca77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:02.202346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2d618-eea5-4cb6-9981-85f5e0e0ca77 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:02.216881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:02.218671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:02.234611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:02.235731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] PATH_INFO: `/v3/users/dc285681bd4843028f0fbbbd01ff4adb` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:02.237651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc285681bd4843028f0fbbbd01ff4adb)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:02.241338 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:02.478553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc285681bd4843028f0fbbbd01ff4adb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:02.576623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc285681bd4843028f0fbbbd01ff4adb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:02.582036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc285681bd4843028f0fbbbd01ff4adb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:02.590424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc285681bd4843028f0fbbbd01ff4adb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:02.596598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636ddf0-6f5b-49cf-8e14-9aeb3e4e4a7a service nova] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:02.615141 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636ddf0-6f5b-49cf-8e14-9aeb3e4e4a7a service nova] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:02.624282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc285681bd4843028f0fbbbd01ff4adb'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:02.625350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d9797-2571-4520-bc4c-e9ac26d63026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:02.627267 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d9797-2571-4520-bc4c-e9ac26d63026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:02.627628 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d9797-2571-4520-bc4c-e9ac26d63026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:02.636386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d9797-2571-4520-bc4c-e9ac26d63026 admin admin] PATH_INFO: `/v3/users/59dc7e2496a24e57aa923173369b5d99` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:02.636386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d9797-2571-4520-bc4c-e9ac26d63026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59dc7e2496a24e57aa923173369b5d99)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:02.636386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d9797-2571-4520-bc4c-e9ac26d63026 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:02.636953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1269/2509] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:56:02 2022] DELETE /identity/v3/users/dc285681bd4843028f0fbbbd01ff4adb => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:02.852597 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9e1d9797-2571-4520-bc4c-e9ac26d63026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59dc7e2496a24e57aa923173369b5d99'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:02.950498 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9e1d9797-2571-4520-bc4c-e9ac26d63026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59dc7e2496a24e57aa923173369b5d99'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:02.951500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9e1d9797-2571-4520-bc4c-e9ac26d63026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59dc7e2496a24e57aa923173369b5d99'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:02.976353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9e1d9797-2571-4520-bc4c-e9ac26d63026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59dc7e2496a24e57aa923173369b5d99'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:02.991579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9e1d9797-2571-4520-bc4c-e9ac26d63026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59dc7e2496a24e57aa923173369b5d99'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:03.007849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:03.009348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:03.012093 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1241/2510] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:56:02 2022] DELETE /identity/v3/users/59dc7e2496a24e57aa923173369b5d99 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:03.018744 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7223295-d8c6-41d1-8a91-62e43b4ad76a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:03.019036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7223295-d8c6-41d1-8a91-62e43b4ad76a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:03.019358 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7223295-d8c6-41d1-8a91-62e43b4ad76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:03.068808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b7223295-d8c6-41d1-8a91-62e43b4ad76a None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:03.070265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b7223295-d8c6-41d1-8a91-62e43b4ad76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:03.120563 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:03.170183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fc0ba-905b-4d9b-bdba-90d773d6b2c7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:03.250286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1242/2511] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:03.261698 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7223295-d8c6-41d1-8a91-62e43b4ad76a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:03.263577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b7223295-d8c6-41d1-8a91-62e43b4ad76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:03.280888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:03.282252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:03.283074 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:03.283573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] PATH_INFO: `/v3/projects/6efde3fd7658438cad16413a10503384` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:03.285513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6efde3fd7658438cad16413a10503384)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:03.289379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:03.340286 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7223295-d8c6-41d1-8a91-62e43b4ad76a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:03.344451 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c28851-ef6a-47e0-acc6-a0d46237a624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:03.345600 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c28851-ef6a-47e0-acc6-a0d46237a624 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:03.345879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c28851-ef6a-47e0-acc6-a0d46237a624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:03.346146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c28851-ef6a-47e0-acc6-a0d46237a624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:03.357403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c28851-ef6a-47e0-acc6-a0d46237a624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:03.359042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c28851-ef6a-47e0-acc6-a0d46237a624 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:03.370928 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1243/2512] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:03.626869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6efde3fd7658438cad16413a10503384', 'request_id': 'req-370dd730-d315-490a-b168-2a84bc548508'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:03.637306 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6efde3fd7658438cad16413a10503384', 'request_id': 'req-370dd730-d315-490a-b168-2a84bc548508'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:03.650413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6efde3fd7658438cad16413a10503384', 'request_id': 'req-370dd730-d315-490a-b168-2a84bc548508'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:03.768760 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] The token cache is being invalidate because project 6efde3fd7658438cad16413a10503384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:56:03.769051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:03.770257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:03.773416 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1270/2513] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:56:03 2022] DELETE /identity/v3/projects/6efde3fd7658438cad16413a10503384 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:56:03.778053 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c28851-ef6a-47e0-acc6-a0d46237a624 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:03.783091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-44c28851-ef6a-47e0-acc6-a0d46237a624 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:03.800235 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:03.818531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c28851-ef6a-47e0-acc6-a0d46237a624 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:03.924120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:03.925150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:03.925150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:03.925329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] PATH_INFO: `/v3/users/4d941fd5e5fd4eb281fc92a71c32d6f9` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:03.926514 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d941fd5e5fd4eb281fc92a71c32d6f9)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:03.928463 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:04.080952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:04.082964 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:04.108428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dd730-d315-490a-b168-2a84bc548508 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:04.121252 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:04.122076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:04.122367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:04.122644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] PATH_INFO: `/v3/projects/59233f3b59524631888657bc9f402e12` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:04.124501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] RBAC: Authorizing `identity:delete_project(project_id=59233f3b59524631888657bc9f402e12)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:04.131167 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:04.166445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d941fd5e5fd4eb281fc92a71c32d6f9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.270401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d941fd5e5fd4eb281fc92a71c32d6f9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.270640 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d941fd5e5fd4eb281fc92a71c32d6f9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.282407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d941fd5e5fd4eb281fc92a71c32d6f9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.310775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59233f3b59524631888657bc9f402e12', 'request_id': 'req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.312878 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d941fd5e5fd4eb281fc92a71c32d6f9'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.325452 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1244/2514] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:56:03 2022] DELETE /identity/v3/users/4d941fd5e5fd4eb281fc92a71c32d6f9 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:04.330550 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59233f3b59524631888657bc9f402e12', 'request_id': 'req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.342315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59233f3b59524631888657bc9f402e12', 'request_id': 'req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.437586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] The token cache is being invalidate because project 59233f3b59524631888657bc9f402e12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:56:04.437940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.439476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.442890 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1271/2515] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:56:04 2022] DELETE /identity/v3/projects/59233f3b59524631888657bc9f402e12 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:04.446725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:04.448269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:04.574049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fa11a-b449-40f1-8ff9-f5204638d65f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:04.581868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:04.582938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:04.583288 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:04.583627 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] PATH_INFO: `/v3/users/84d30adb5a4d4824b4db3a8884f59b69` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:04.585130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d30adb5a4d4824b4db3a8884f59b69)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:04.587927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:04.633057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:04.805922 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:04.821984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:04.825962 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d30adb5a4d4824b4db3a8884f59b69'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.862229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:04.898456 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d30adb5a4d4824b4db3a8884f59b69'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.899203 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d30adb5a4d4824b4db3a8884f59b69'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.927883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d30adb5a4d4824b4db3a8884f59b69'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.952340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d30adb5a4d4824b4db3a8884f59b69'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:04.966371 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1245/2516] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:56:04 2022] DELETE /identity/v3/users/84d30adb5a4d4824b4db3a8884f59b69 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:05.004575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c87a59-bf6a-41fe-a5b1-3b0eaeb9682d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:05.074001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc4f38-9df2-4478-b5af-fb8cba66c215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:05.074913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc4f38-9df2-4478-b5af-fb8cba66c215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:05.075362 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc4f38-9df2-4478-b5af-fb8cba66c215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:05.075823 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc4f38-9df2-4478-b5af-fb8cba66c215 admin admin] PATH_INFO: `/v3/projects/4d6e861a742c41c488ab1a4060f63fc9` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:05.077108 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc4f38-9df2-4478-b5af-fb8cba66c215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d6e861a742c41c488ab1a4060f63fc9)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:05.080114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc4f38-9df2-4478-b5af-fb8cba66c215 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:05.242717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-94dc4f38-9df2-4478-b5af-fb8cba66c215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d6e861a742c41c488ab1a4060f63fc9', 'request_id': 'req-94dc4f38-9df2-4478-b5af-fb8cba66c215'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:05.261330 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-94dc4f38-9df2-4478-b5af-fb8cba66c215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d6e861a742c41c488ab1a4060f63fc9', 'request_id': 'req-94dc4f38-9df2-4478-b5af-fb8cba66c215'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:05.281149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-94dc4f38-9df2-4478-b5af-fb8cba66c215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d6e861a742c41c488ab1a4060f63fc9', 'request_id': 'req-94dc4f38-9df2-4478-b5af-fb8cba66c215'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:05.379991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-94dc4f38-9df2-4478-b5af-fb8cba66c215 admin admin] The token cache is being invalidate because project 4d6e861a742c41c488ab1a4060f63fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:56:05.381226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-94dc4f38-9df2-4478-b5af-fb8cba66c215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:05.381226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-94dc4f38-9df2-4478-b5af-fb8cba66c215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:05.382980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1272/2517] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:56:04 2022] DELETE /identity/v3/projects/4d6e861a742c41c488ab1a4060f63fc9 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:05.588029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:05.589560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:05.612384 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d990c-bc28-4591-80d1-34c837119327 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:05.825769 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:05.826608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:05.827492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:05.828023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6 admin admin] PATH_INFO: `/v3/projects/569b9892afde4f89b6573fd62183d929` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:05.829639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=569b9892afde4f89b6573fd62183d929)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:05.833513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:06.014866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '569b9892afde4f89b6573fd62183d929', 'request_id': 'req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:06.017152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '569b9892afde4f89b6573fd62183d929', 'request_id': 'req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:06.028574 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '569b9892afde4f89b6573fd62183d929', 'request_id': 'req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:06.083630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6 admin admin] The token cache is being invalidate because project 569b9892afde4f89b6573fd62183d929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:56:06.084316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:06.085113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-aa2f7023-01a1-4c10-9a63-88e1aa1de7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:06.096679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1246/2518] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:56:05 2022] DELETE /identity/v3/projects/569b9892afde4f89b6573fd62183d929 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:06.098277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba8e39-d9fc-4e9e-8ba4-b08efb71c96c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:06.098664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba8e39-d9fc-4e9e-8ba4-b08efb71c96c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:06.099023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba8e39-d9fc-4e9e-8ba4-b08efb71c96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:06.137651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d8ba8e39-d9fc-4e9e-8ba4-b08efb71c96c None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:06.138951 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d8ba8e39-d9fc-4e9e-8ba4-b08efb71c96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:06.216651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:06.250244 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1273/2519] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:56:06.252211 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728fa5a-2718-46e7-bcf5-64164dd0c065 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:06.254718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728fa5a-2718-46e7-bcf5-64164dd0c065 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:06.254718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728fa5a-2718-46e7-bcf5-64164dd0c065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:06.300534 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1728fa5a-2718-46e7-bcf5-64164dd0c065 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:06.303916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1728fa5a-2718-46e7-bcf5-64164dd0c065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:06.325006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1247/2520] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:06.329409 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba8e39-d9fc-4e9e-8ba4-b08efb71c96c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:06.353309 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba8e39-d9fc-4e9e-8ba4-b08efb71c96c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:06.362291 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe937fb-0342-478b-af66-5e8a3b8c20b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:06.362910 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe937fb-0342-478b-af66-5e8a3b8c20b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:06.363186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe937fb-0342-478b-af66-5e8a3b8c20b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:06.363452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe937fb-0342-478b-af66-5e8a3b8c20b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:06.364275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe937fb-0342-478b-af66-5e8a3b8c20b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:56:06.364740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe937fb-0342-478b-af66-5e8a3b8c20b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:06.366679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe937fb-0342-478b-af66-5e8a3b8c20b7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:06.391642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1274/2521] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:56:06.396006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728fa5a-2718-46e7-bcf5-64164dd0c065 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:06.410903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728fa5a-2718-46e7-bcf5-64164dd0c065 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:06.416940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5cad3-848a-4b4a-80b9-dd1c67f2d562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:06.417675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5cad3-848a-4b4a-80b9-dd1c67f2d562 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:06.418026 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5cad3-848a-4b4a-80b9-dd1c67f2d562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:06.418378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5cad3-848a-4b4a-80b9-dd1c67f2d562 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:06.419098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5cad3-848a-4b4a-80b9-dd1c67f2d562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:06.421082 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5cad3-848a-4b4a-80b9-dd1c67f2d562 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:06.571903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1248/2522] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:56:06 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:06.575581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe937fb-0342-478b-af66-5e8a3b8c20b7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:06.590571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe937fb-0342-478b-af66-5e8a3b8c20b7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:06.696452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2310b0-e814-4fe3-8009-e5cddcd9f876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:06.697347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2310b0-e814-4fe3-8009-e5cddcd9f876 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:06.703923 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2310b0-e814-4fe3-8009-e5cddcd9f876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:06.703923 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2310b0-e814-4fe3-8009-e5cddcd9f876 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:06.703923 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2310b0-e814-4fe3-8009-e5cddcd9f876 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:06.703923 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2310b0-e814-4fe3-8009-e5cddcd9f876 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:06.790847 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1275/2523] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:56:06 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:06.794778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5cad3-848a-4b4a-80b9-dd1c67f2d562 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:06.802064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:06.814172 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5cad3-848a-4b4a-80b9-dd1c67f2d562 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:06.821681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447708c3-5fa9-4cd7-90db-db9247c103bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:06.822995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447708c3-5fa9-4cd7-90db-db9247c103bc admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:06.823791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447708c3-5fa9-4cd7-90db-db9247c103bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:06.824529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447708c3-5fa9-4cd7-90db-db9247c103bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:06.826047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447708c3-5fa9-4cd7-90db-db9247c103bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:06.830284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447708c3-5fa9-4cd7-90db-db9247c103bc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:06.846806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1249/2524] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:06.850972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2310b0-e814-4fe3-8009-e5cddcd9f876 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:06.868076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2310b0-e814-4fe3-8009-e5cddcd9f876 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:06.873224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24caafa1-9e1e-46ab-809e-a1c830f9309a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:06.875723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24caafa1-9e1e-46ab-809e-a1c830f9309a admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:06.876891 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24caafa1-9e1e-46ab-809e-a1c830f9309a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:06.877316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24caafa1-9e1e-46ab-809e-a1c830f9309a admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:06.886146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24caafa1-9e1e-46ab-809e-a1c830f9309a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:06.886146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24caafa1-9e1e-46ab-809e-a1c830f9309a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:06.902787 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447708c3-5fa9-4cd7-90db-db9247c103bc admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:06.914717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1276/2525] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:06.917019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447708c3-5fa9-4cd7-90db-db9247c103bc admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:06.924805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e60ff-1cbf-4f3d-9e03-be7e6c27e0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:06.926188 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e60ff-1cbf-4f3d-9e03-be7e6c27e0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:06.926578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e60ff-1cbf-4f3d-9e03-be7e6c27e0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:06.926958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e60ff-1cbf-4f3d-9e03-be7e6c27e0eb admin admin] PATH_INFO: `/v3/projects/7e81a89ed40c43f28ac96cbe3744e6c0/users/aa415c4e0e9c4928b51152fa0a428d9b/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:06.951733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e60ff-1cbf-4f3d-9e03-be7e6c27e0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e81a89ed40c43f28ac96cbe3744e6c0, user_id=aa415c4e0e9c4928b51152fa0a428d9b, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:06.959004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e60ff-1cbf-4f3d-9e03-be7e6c27e0eb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:07.050523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24caafa1-9e1e-46ab-809e-a1c830f9309a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:07.056675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-24caafa1-9e1e-46ab-809e-a1c830f9309a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:07.056819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1250/2526] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:06 2022] PUT /identity/v3/projects/7e81a89ed40c43f28ac96cbe3744e6c0/users/aa415c4e0e9c4928b51152fa0a428d9b/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:07.061890 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346d319-ad47-42f4-b384-0abedefb0e5e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:07.062495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346d319-ad47-42f4-b384-0abedefb0e5e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:07.063034 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346d319-ad47-42f4-b384-0abedefb0e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:07.075649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24caafa1-9e1e-46ab-809e-a1c830f9309a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:07.092319 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d346d319-ad47-42f4-b384-0abedefb0e5e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:07.093584 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d346d319-ad47-42f4-b384-0abedefb0e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:07.161582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb09ce8-81e3-4e60-a76a-bdfaba602cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:07.162373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb09ce8-81e3-4e60-a76a-bdfaba602cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:07.162795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb09ce8-81e3-4e60-a76a-bdfaba602cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:07.163077 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb09ce8-81e3-4e60-a76a-bdfaba602cd3 admin admin] PATH_INFO: `/v3/users/ef8af9fd9e844ca4be10caa880357a4f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:07.164238 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb09ce8-81e3-4e60-a76a-bdfaba602cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef8af9fd9e844ca4be10caa880357a4f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:07.166346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb09ce8-81e3-4e60-a76a-bdfaba602cd3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:07.169602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1251/2527] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:56:07.176490 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346d319-ad47-42f4-b384-0abedefb0e5e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:07.177922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d346d319-ad47-42f4-b384-0abedefb0e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:07.243713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346d319-ad47-42f4-b384-0abedefb0e5e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:07.247597 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4373d7-0d9e-4e57-8ff6-686f0744022b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:07.248601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4373d7-0d9e-4e57-8ff6-686f0744022b service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:07.248601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4373d7-0d9e-4e57-8ff6-686f0744022b service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:07.248819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4373d7-0d9e-4e57-8ff6-686f0744022b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:07.259644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4373d7-0d9e-4e57-8ff6-686f0744022b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:07.261112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4373d7-0d9e-4e57-8ff6-686f0744022b service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:07.273374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1252/2528] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:56:07.388650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3fb09ce8-81e3-4e60-a76a-bdfaba602cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef8af9fd9e844ca4be10caa880357a4f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:07.475881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3fb09ce8-81e3-4e60-a76a-bdfaba602cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef8af9fd9e844ca4be10caa880357a4f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:07.476521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3fb09ce8-81e3-4e60-a76a-bdfaba602cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef8af9fd9e844ca4be10caa880357a4f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:07.483276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:07.487579 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3fb09ce8-81e3-4e60-a76a-bdfaba602cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef8af9fd9e844ca4be10caa880357a4f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:07.516701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-3fb09ce8-81e3-4e60-a76a-bdfaba602cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef8af9fd9e844ca4be10caa880357a4f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:07.524261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1277/2529] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:56:07 2022] DELETE /identity/v3/users/ef8af9fd9e844ca4be10caa880357a4f => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:56:07.809265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4373d7-0d9e-4e57-8ff6-686f0744022b service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:07.824191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4373d7-0d9e-4e57-8ff6-686f0744022b service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:07.932221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9eb07-e494-4a14-983f-437381f96db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:07.933362 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9eb07-e494-4a14-983f-437381f96db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:07.933864 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9eb07-e494-4a14-983f-437381f96db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:07.934312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9eb07-e494-4a14-983f-437381f96db8 admin admin] PATH_INFO: `/v3/projects/6d738e0a61184b4db58e2607d72cca0a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:07.936784 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9eb07-e494-4a14-983f-437381f96db8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d738e0a61184b4db58e2607d72cca0a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:07.939790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9eb07-e494-4a14-983f-437381f96db8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:07.956402 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:08.418635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-abb9eb07-e494-4a14-983f-437381f96db8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d738e0a61184b4db58e2607d72cca0a', 'request_id': 'req-abb9eb07-e494-4a14-983f-437381f96db8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:08.424761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-abb9eb07-e494-4a14-983f-437381f96db8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d738e0a61184b4db58e2607d72cca0a', 'request_id': 'req-abb9eb07-e494-4a14-983f-437381f96db8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:08.431556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-abb9eb07-e494-4a14-983f-437381f96db8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d738e0a61184b4db58e2607d72cca0a', 'request_id': 'req-abb9eb07-e494-4a14-983f-437381f96db8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:08.515050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-abb9eb07-e494-4a14-983f-437381f96db8 admin admin] The token cache is being invalidate because project 6d738e0a61184b4db58e2607d72cca0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:56:08.515420 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-abb9eb07-e494-4a14-983f-437381f96db8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:08.516162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-abb9eb07-e494-4a14-983f-437381f96db8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:08.518363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1253/2530] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:56:07 2022] DELETE /identity/v3/projects/6d738e0a61184b4db58e2607d72cca0a => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:08.523428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462744ee-6d29-45e1-95ef-d37942ca5731 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:08.523945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462744ee-6d29-45e1-95ef-d37942ca5731 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:08.524056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462744ee-6d29-45e1-95ef-d37942ca5731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:08.544821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-462744ee-6d29-45e1-95ef-d37942ca5731 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:08.546062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-462744ee-6d29-45e1-95ef-d37942ca5731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:08.629703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1278/2531] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:56:08.638733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4147f8-c31d-47b1-a6e1-87388ca95b4c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:08.639566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4147f8-c31d-47b1-a6e1-87388ca95b4c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:08.640028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4147f8-c31d-47b1-a6e1-87388ca95b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:08.663887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-7f4147f8-c31d-47b1-a6e1-87388ca95b4c None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:08.665123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-7f4147f8-c31d-47b1-a6e1-87388ca95b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:08.680174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1254/2532] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:08.684036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462744ee-6d29-45e1-95ef-d37942ca5731 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:08.698004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462744ee-6d29-45e1-95ef-d37942ca5731 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:08.703359 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232bf5b-63f0-4be3-886f-a55d82a9a46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:08.704116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232bf5b-63f0-4be3-886f-a55d82a9a46b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:08.705016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232bf5b-63f0-4be3-886f-a55d82a9a46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:08.705467 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232bf5b-63f0-4be3-886f-a55d82a9a46b admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:08.710540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232bf5b-63f0-4be3-886f-a55d82a9a46b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:56:08.710540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232bf5b-63f0-4be3-886f-a55d82a9a46b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:08.710540 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232bf5b-63f0-4be3-886f-a55d82a9a46b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:08.727644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1279/2533] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:56:08.732187 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4147f8-c31d-47b1-a6e1-87388ca95b4c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:08.743465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4147f8-c31d-47b1-a6e1-87388ca95b4c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:08.753252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03258edb-fab8-4f3f-b1ef-e3332f89ccc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:08.753891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03258edb-fab8-4f3f-b1ef-e3332f89ccc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:08.759341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03258edb-fab8-4f3f-b1ef-e3332f89ccc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:08.759341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03258edb-fab8-4f3f-b1ef-e3332f89ccc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:08.759341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03258edb-fab8-4f3f-b1ef-e3332f89ccc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:08.759341 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03258edb-fab8-4f3f-b1ef-e3332f89ccc9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:08.866963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1255/2534] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:56:08 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:08.867730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232bf5b-63f0-4be3-886f-a55d82a9a46b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:08.887381 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232bf5b-63f0-4be3-886f-a55d82a9a46b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:08.984502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bc9fc-0bd7-47da-a9a5-c53c3d52068f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:08.985143 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bc9fc-0bd7-47da-a9a5-c53c3d52068f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:08.986021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bc9fc-0bd7-47da-a9a5-c53c3d52068f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:08.986021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bc9fc-0bd7-47da-a9a5-c53c3d52068f admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:08.986647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bc9fc-0bd7-47da-a9a5-c53c3d52068f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:08.988888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bc9fc-0bd7-47da-a9a5-c53c3d52068f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:09.051690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f9a0c-f23d-46d9-a4aa-93c7c3ea720a None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:09.052450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f9a0c-f23d-46d9-a4aa-93c7c3ea720a None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:09.052727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f9a0c-f23d-46d9-a4aa-93c7c3ea720a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:09.090268 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1280/2535] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:56:08 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:56:09.094627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bc9fc-0bd7-47da-a9a5-c53c3d52068f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:09.119602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bc9fc-0bd7-47da-a9a5-c53c3d52068f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:09.123199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-985f9a0c-f23d-46d9-a4aa-93c7c3ea720a None None] MFA Rules not processed for user `aa415c4e0e9c4928b51152fa0a428d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:09.124473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-985f9a0c-f23d-46d9-a4aa-93c7c3ea720a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:09.133789 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee56637-ec0e-4eca-8efe-7d85c9a815df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:09.134420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee56637-ec0e-4eca-8efe-7d85c9a815df admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:09.134699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee56637-ec0e-4eca-8efe-7d85c9a815df admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:09.134981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee56637-ec0e-4eca-8efe-7d85c9a815df admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:09.135687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee56637-ec0e-4eca-8efe-7d85c9a815df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:09.138101 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee56637-ec0e-4eca-8efe-7d85c9a815df admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:09.153893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1281/2536] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:09.155607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee56637-ec0e-4eca-8efe-7d85c9a815df admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:09.179708 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee56637-ec0e-4eca-8efe-7d85c9a815df admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:09.187607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072590bc-be7e-46e9-a3df-6b8803944731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:09.188492 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072590bc-be7e-46e9-a3df-6b8803944731 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:09.189182 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072590bc-be7e-46e9-a3df-6b8803944731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:09.189418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072590bc-be7e-46e9-a3df-6b8803944731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:09.190051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072590bc-be7e-46e9-a3df-6b8803944731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:09.191798 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072590bc-be7e-46e9-a3df-6b8803944731 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:09.209250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1282/2537] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:09.210396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072590bc-be7e-46e9-a3df-6b8803944731 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:09.231017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072590bc-be7e-46e9-a3df-6b8803944731 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:09.238559 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e6416-2968-40d9-8cea-43d263f1535f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:09.239885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e6416-2968-40d9-8cea-43d263f1535f admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:09.239885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e6416-2968-40d9-8cea-43d263f1535f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:09.240257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e6416-2968-40d9-8cea-43d263f1535f admin admin] PATH_INFO: `/v3/projects/c2d43a17af3444f48961b49b5442613f/users/1c4a100a2ce140f4b39cd224ecf18c24/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:09.271630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e6416-2968-40d9-8cea-43d263f1535f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d43a17af3444f48961b49b5442613f, user_id=1c4a100a2ce140f4b39cd224ecf18c24, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:09.278118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1256/2538] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:09.282152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e6416-2968-40d9-8cea-43d263f1535f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:09.294156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f9a0c-f23d-46d9-a4aa-93c7c3ea720a None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:09.295863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-985f9a0c-f23d-46d9-a4aa-93c7c3ea720a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:09.351838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1283/2539] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:09 2022] PUT /identity/v3/projects/c2d43a17af3444f48961b49b5442613f/users/1c4a100a2ce140f4b39cd224ecf18c24/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:09.353380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd0854-df48-4bea-886d-a9597b2a49cc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:09.353692 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd0854-df48-4bea-886d-a9597b2a49cc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:09.354326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd0854-df48-4bea-886d-a9597b2a49cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:09.373824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f9a0c-f23d-46d9-a4aa-93c7c3ea720a None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:09.383852 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:09.390282 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:09.423004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-63dd0854-df48-4bea-886d-a9597b2a49cc None None] MFA Rules not processed for user `1c4a100a2ce140f4b39cd224ecf18c24`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:09.424305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-63dd0854-df48-4bea-886d-a9597b2a49cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:09.459262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e610d-08c9-4b99-9074-8bed8db62b6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:09.459882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e610d-08c9-4b99-9074-8bed8db62b6f service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:09.460168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e610d-08c9-4b99-9074-8bed8db62b6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:09.460429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e610d-08c9-4b99-9074-8bed8db62b6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:09.516173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1284/2540] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:09.527950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd0854-df48-4bea-886d-a9597b2a49cc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:09.530217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-63dd0854-df48-4bea-886d-a9597b2a49cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:09.537329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e610d-08c9-4b99-9074-8bed8db62b6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:09.538615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e610d-08c9-4b99-9074-8bed8db62b6f service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:09.549937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1257/2541] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:56:09.617401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd0854-df48-4bea-886d-a9597b2a49cc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:09.625716 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2969e-8668-4fa0-a043-5b7287ebc615 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:09.626902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2969e-8668-4fa0-a043-5b7287ebc615 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:09.627174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2969e-8668-4fa0-a043-5b7287ebc615 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:09.627430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2969e-8668-4fa0-a043-5b7287ebc615 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:09.637187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2969e-8668-4fa0-a043-5b7287ebc615 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:09.638089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2969e-8668-4fa0-a043-5b7287ebc615 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:09.658746 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1285/2542] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:11.399131 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfc6f7-a35f-4180-b00c-c040ddcfd1d8 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:11.399569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfc6f7-a35f-4180-b00c-c040ddcfd1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:11.399694 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfc6f7-a35f-4180-b00c-c040ddcfd1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:11.419299 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-a6dfc6f7-a35f-4180-b00c-c040ddcfd1d8 None None] MFA Rules not processed for user `aa415c4e0e9c4928b51152fa0a428d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:11.420580 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a6dfc6f7-a35f-4180-b00c-c040ddcfd1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:11.427609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1258/2543] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:12.617226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e14f2-90db-4dab-abaa-9f0ef461bb0a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:12.617773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e14f2-90db-4dab-abaa-9f0ef461bb0a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:12.617773 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e14f2-90db-4dab-abaa-9f0ef461bb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:12.637276 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-bc5e14f2-90db-4dab-abaa-9f0ef461bb0a None None] MFA Rules not processed for user `db26f48da0af4723afb2c0a4c5f52d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:12.638525 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-bc5e14f2-90db-4dab-abaa-9f0ef461bb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:12.702447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1286/2544] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:18.267906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfc6f7-a35f-4180-b00c-c040ddcfd1d8 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:18.342849 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfc6f7-a35f-4180-b00c-c040ddcfd1d8 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:18.348830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf84a8-cb21-4404-95b1-07283ed07b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:18.349557 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf84a8-cb21-4404-95b1-07283ed07b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:18.349937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf84a8-cb21-4404-95b1-07283ed07b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:18.350281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf84a8-cb21-4404-95b1-07283ed07b79 admin admin] PATH_INFO: `/v3/users/1c4a100a2ce140f4b39cd224ecf18c24` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:18.352081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf84a8-cb21-4404-95b1-07283ed07b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c4a100a2ce140f4b39cd224ecf18c24)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:18.354505 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf84a8-cb21-4404-95b1-07283ed07b79 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:18.472485 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4ee8a-febe-4015-963f-62adf9023f24 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:18.473000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4ee8a-febe-4015-963f-62adf9023f24 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:18.473257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4ee8a-febe-4015-963f-62adf9023f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:18.496170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5cf4ee8a-febe-4015-963f-62adf9023f24 None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:18.497853 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5cf4ee8a-febe-4015-963f-62adf9023f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:18.528625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:18.572980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4caf84a8-cb21-4404-95b1-07283ed07b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c4a100a2ce140f4b39cd224ecf18c24'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:18.580219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1287/2545] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:56:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:56:18.595058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4ee8a-febe-4015-963f-62adf9023f24 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:18.596490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5cf4ee8a-febe-4015-963f-62adf9023f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:18.652985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4caf84a8-cb21-4404-95b1-07283ed07b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c4a100a2ce140f4b39cd224ecf18c24'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:18.655215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4caf84a8-cb21-4404-95b1-07283ed07b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c4a100a2ce140f4b39cd224ecf18c24'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:18.670927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4caf84a8-cb21-4404-95b1-07283ed07b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c4a100a2ce140f4b39cd224ecf18c24'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:18.679761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:18.682701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4ee8a-febe-4015-963f-62adf9023f24 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:18.689328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ba225-49bc-47a1-9518-e97b847692dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:18.689988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ba225-49bc-47a1-9518-e97b847692dd service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:18.690271 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ba225-49bc-47a1-9518-e97b847692dd service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:18.690673 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ba225-49bc-47a1-9518-e97b847692dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:18.695664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3d3ba225-49bc-47a1-9518-e97b847692dd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:18.695839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4caf84a8-cb21-4404-95b1-07283ed07b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c4a100a2ce140f4b39cd224ecf18c24'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:18.707058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1259/2546] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:56:18 2022] DELETE /identity/v3/users/1c4a100a2ce140f4b39cd224ecf18c24 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:18.709078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c0346-2ad5-4289-a996-09491f14dac5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:18.709462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c0346-2ad5-4289-a996-09491f14dac5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:18.710377 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c0346-2ad5-4289-a996-09491f14dac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:18.712483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ba225-49bc-47a1-9518-e97b847692dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:18.715954 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ba225-49bc-47a1-9518-e97b847692dd service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:18.738973 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b80c0346-2ad5-4289-a996-09491f14dac5 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:18.740400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b80c0346-2ad5-4289-a996-09491f14dac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:18.750013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1288/2547] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:56:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 18:56:18.840739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1260/2548] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:18.883770 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ba225-49bc-47a1-9518-e97b847692dd service swift] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:18.883770 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3d3ba225-49bc-47a1-9518-e97b847692dd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:18.961403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ba225-49bc-47a1-9518-e97b847692dd service swift] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:19.040218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854eac3-8523-4fcd-8d08-280b9c74fbcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:19.040996 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854eac3-8523-4fcd-8d08-280b9c74fbcb service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:19.041388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854eac3-8523-4fcd-8d08-280b9c74fbcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:19.041759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854eac3-8523-4fcd-8d08-280b9c74fbcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:19.059289 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854eac3-8523-4fcd-8d08-280b9c74fbcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:19.061772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854eac3-8523-4fcd-8d08-280b9c74fbcb service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:19.076977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1289/2549] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:20.792508 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c0346-2ad5-4289-a996-09491f14dac5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:20.794353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b80c0346-2ad5-4289-a996-09491f14dac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:20.809493 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c0346-2ad5-4289-a996-09491f14dac5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:20.816306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43d212-e603-4c13-bef4-ce10f3791374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:20.816973 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43d212-e603-4c13-bef4-ce10f3791374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:20.817325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43d212-e603-4c13-bef4-ce10f3791374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:20.817821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43d212-e603-4c13-bef4-ce10f3791374 admin admin] PATH_INFO: `/v3/projects/c2d43a17af3444f48961b49b5442613f` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:20.819644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43d212-e603-4c13-bef4-ce10f3791374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2d43a17af3444f48961b49b5442613f)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:20.821252 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43d212-e603-4c13-bef4-ce10f3791374 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:21.028099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9d43d212-e603-4c13-bef4-ce10f3791374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2d43a17af3444f48961b49b5442613f', 'request_id': 'req-9d43d212-e603-4c13-bef4-ce10f3791374'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:21.045076 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9d43d212-e603-4c13-bef4-ce10f3791374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2d43a17af3444f48961b49b5442613f', 'request_id': 'req-9d43d212-e603-4c13-bef4-ce10f3791374'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:21.061155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9d43d212-e603-4c13-bef4-ce10f3791374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2d43a17af3444f48961b49b5442613f', 'request_id': 'req-9d43d212-e603-4c13-bef4-ce10f3791374'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:21.146632 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9d43d212-e603-4c13-bef4-ce10f3791374 admin admin] The token cache is being invalidate because project c2d43a17af3444f48961b49b5442613f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:56:21.147275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9d43d212-e603-4c13-bef4-ce10f3791374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:21.148157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-9d43d212-e603-4c13-bef4-ce10f3791374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:21.149956 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1261/2550] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:56:20 2022] DELETE /identity/v3/projects/c2d43a17af3444f48961b49b5442613f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:21.154978 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3bc7d-47be-4cfa-a0cb-fb792f8dc6db None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:21.155986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3bc7d-47be-4cfa-a0cb-fb792f8dc6db None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:21.156466 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3bc7d-47be-4cfa-a0cb-fb792f8dc6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:21.184882 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f4e3bc7d-47be-4cfa-a0cb-fb792f8dc6db None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:21.186677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f4e3bc7d-47be-4cfa-a0cb-fb792f8dc6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:21.211307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:21.285720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1290/2551] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:21.295178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75117869-6d1d-40f2-9467-9dd977df8560 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:21.296426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75117869-6d1d-40f2-9467-9dd977df8560 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:21.296426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75117869-6d1d-40f2-9467-9dd977df8560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:21.339727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-75117869-6d1d-40f2-9467-9dd977df8560 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:21.341360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-75117869-6d1d-40f2-9467-9dd977df8560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:21.353575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1262/2552] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:21.357846 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3bc7d-47be-4cfa-a0cb-fb792f8dc6db None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:21.370387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3bc7d-47be-4cfa-a0cb-fb792f8dc6db None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:21.375759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44a598-81a8-4063-917b-3221fceb32c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:21.376505 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44a598-81a8-4063-917b-3221fceb32c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:21.376916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44a598-81a8-4063-917b-3221fceb32c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:21.377195 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44a598-81a8-4063-917b-3221fceb32c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:21.378430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44a598-81a8-4063-917b-3221fceb32c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:56:21.378430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44a598-81a8-4063-917b-3221fceb32c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:21.384222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44a598-81a8-4063-917b-3221fceb32c8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:21.394087 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1291/2553] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:56:21.398938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75117869-6d1d-40f2-9467-9dd977df8560 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:21.412242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75117869-6d1d-40f2-9467-9dd977df8560 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:21.422258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf5fb9-e6a0-42c1-b510-f2e031781333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:21.423058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf5fb9-e6a0-42c1-b510-f2e031781333 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:21.423361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf5fb9-e6a0-42c1-b510-f2e031781333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:21.425807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf5fb9-e6a0-42c1-b510-f2e031781333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:21.427244 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf5fb9-e6a0-42c1-b510-f2e031781333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:21.436210 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf5fb9-e6a0-42c1-b510-f2e031781333 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:21.589776 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1263/2554] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:56:21 2022] POST /identity/v3/projects => generated 373 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:56:21.594103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44a598-81a8-4063-917b-3221fceb32c8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:21.605932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44a598-81a8-4063-917b-3221fceb32c8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:21.760587 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad66efb-ba70-4b2f-a568-93d11f5b347b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:21.761404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad66efb-ba70-4b2f-a568-93d11f5b347b admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:21.761938 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad66efb-ba70-4b2f-a568-93d11f5b347b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:21.762336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad66efb-ba70-4b2f-a568-93d11f5b347b admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:21.763331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad66efb-ba70-4b2f-a568-93d11f5b347b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:21.766286 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad66efb-ba70-4b2f-a568-93d11f5b347b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:21.886438 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1292/2555] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:56:21 2022] POST /identity/v3/users => generated 287 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:21.887824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf5fb9-e6a0-42c1-b510-f2e031781333 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:21.910023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf5fb9-e6a0-42c1-b510-f2e031781333 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:21.921904 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11f24b-34d2-4906-959e-37247584ebd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:21.925230 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11f24b-34d2-4906-959e-37247584ebd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:21.927938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11f24b-34d2-4906-959e-37247584ebd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:21.928917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11f24b-34d2-4906-959e-37247584ebd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:21.931133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11f24b-34d2-4906-959e-37247584ebd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:21.937454 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11f24b-34d2-4906-959e-37247584ebd3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:21.968140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1264/2556] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:21.972239 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad66efb-ba70-4b2f-a568-93d11f5b347b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:21.988134 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad66efb-ba70-4b2f-a568-93d11f5b347b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:21.996760 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b14c8-0745-4f74-b1b6-c142c3851e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:21.997541 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b14c8-0745-4f74-b1b6-c142c3851e9c admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:21.997929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b14c8-0745-4f74-b1b6-c142c3851e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:21.998273 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b14c8-0745-4f74-b1b6-c142c3851e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:21.999089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b14c8-0745-4f74-b1b6-c142c3851e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:22.000907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b14c8-0745-4f74-b1b6-c142c3851e9c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:22.026173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1293/2557] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:22.031336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11f24b-34d2-4906-959e-37247584ebd3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:22.050485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11f24b-34d2-4906-959e-37247584ebd3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:22.079568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e639f-7811-48f7-ba06-80765bfa8ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:22.079568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e639f-7811-48f7-ba06-80765bfa8ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:22.079568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e639f-7811-48f7-ba06-80765bfa8ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:22.079568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e639f-7811-48f7-ba06-80765bfa8ce3 admin admin] PATH_INFO: `/v3/projects/5460728e12e748eeab5f345b25177f62/users/653103ee529a4f4c8e72b0bc9be5df13/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:22.100218 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e639f-7811-48f7-ba06-80765bfa8ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5460728e12e748eeab5f345b25177f62, user_id=653103ee529a4f4c8e72b0bc9be5df13, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:22.106717 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e639f-7811-48f7-ba06-80765bfa8ce3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:22.195322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1265/2558] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:22 2022] PUT /identity/v3/projects/5460728e12e748eeab5f345b25177f62/users/653103ee529a4f4c8e72b0bc9be5df13/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:22.206967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458cb41-ee16-441b-b79a-94f5be2977e4 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:22.208307 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458cb41-ee16-441b-b79a-94f5be2977e4 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:22.209345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458cb41-ee16-441b-b79a-94f5be2977e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:22.272447 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8458cb41-ee16-441b-b79a-94f5be2977e4 None None] MFA Rules not processed for user `653103ee529a4f4c8e72b0bc9be5df13`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:22.275271 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8458cb41-ee16-441b-b79a-94f5be2977e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:22.446234 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1294/2559] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:22.450526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e639f-7811-48f7-ba06-80765bfa8ce3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:22.452566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3d5e639f-7811-48f7-ba06-80765bfa8ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:22.462662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:22.578722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e639f-7811-48f7-ba06-80765bfa8ce3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:22.584994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b2087-14f2-41f9-b550-9038649f5776 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:22.585739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b2087-14f2-41f9-b550-9038649f5776 service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:22.586155 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b2087-14f2-41f9-b550-9038649f5776 service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:22.586412 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b2087-14f2-41f9-b550-9038649f5776 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:22.598196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b2087-14f2-41f9-b550-9038649f5776 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:22.604324 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b2087-14f2-41f9-b550-9038649f5776 service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:22.611326 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1266/2560] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:56:22.903972 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458cb41-ee16-441b-b79a-94f5be2977e4 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:22.941124 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:23.006147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458cb41-ee16-441b-b79a-94f5be2977e4 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:23.012147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:23.013366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:23.014690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:23.026089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] PATH_INFO: `/v3/users/653103ee529a4f4c8e72b0bc9be5df13` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:23.026089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=653103ee529a4f4c8e72b0bc9be5df13)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:23.026089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:23.222549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '653103ee529a4f4c8e72b0bc9be5df13'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:23.327670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '653103ee529a4f4c8e72b0bc9be5df13'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:23.328310 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '653103ee529a4f4c8e72b0bc9be5df13'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:23.337703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '653103ee529a4f4c8e72b0bc9be5df13'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:23.351915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '653103ee529a4f4c8e72b0bc9be5df13'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:23.361964 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1295/2561] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:56:22 2022] DELETE /identity/v3/users/653103ee529a4f4c8e72b0bc9be5df13 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:23.363461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f75450-8d12-49e5-ba6c-f85a7b8b9718 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:23.366764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f75450-8d12-49e5-ba6c-f85a7b8b9718 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:23.366764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f75450-8d12-49e5-ba6c-f85a7b8b9718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:23.406313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-74f75450-8d12-49e5-ba6c-f85a7b8b9718 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:23.410784 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-74f75450-8d12-49e5-ba6c-f85a7b8b9718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:23.522844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1267/2562] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:23.529751 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:23.531537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:23.602226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b2668-5bc3-4abf-9930-de9977040595 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:23.606365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55a348-069e-4c08-950f-84086738eea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:23.606986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55a348-069e-4c08-950f-84086738eea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:23.607266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55a348-069e-4c08-950f-84086738eea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:23.607555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55a348-069e-4c08-950f-84086738eea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:23.617763 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55a348-069e-4c08-950f-84086738eea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:23.619279 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55a348-069e-4c08-950f-84086738eea4 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:23.631257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1296/2563] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:24.437689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f75450-8d12-49e5-ba6c-f85a7b8b9718 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:24.439664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-74f75450-8d12-49e5-ba6c-f85a7b8b9718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:24.461050 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f75450-8d12-49e5-ba6c-f85a7b8b9718 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:24.471350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bff86b-8747-42bf-9cf2-d2ec5d88876c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:24.471350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bff86b-8747-42bf-9cf2-d2ec5d88876c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:24.471350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bff86b-8747-42bf-9cf2-d2ec5d88876c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:24.471350 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bff86b-8747-42bf-9cf2-d2ec5d88876c admin admin] PATH_INFO: `/v3/projects/5460728e12e748eeab5f345b25177f62` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:24.474283 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bff86b-8747-42bf-9cf2-d2ec5d88876c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5460728e12e748eeab5f345b25177f62)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:24.486655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bff86b-8747-42bf-9cf2-d2ec5d88876c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:24.698016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-54bff86b-8747-42bf-9cf2-d2ec5d88876c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5460728e12e748eeab5f345b25177f62', 'request_id': 'req-54bff86b-8747-42bf-9cf2-d2ec5d88876c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:24.705367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-54bff86b-8747-42bf-9cf2-d2ec5d88876c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5460728e12e748eeab5f345b25177f62', 'request_id': 'req-54bff86b-8747-42bf-9cf2-d2ec5d88876c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:24.712974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-54bff86b-8747-42bf-9cf2-d2ec5d88876c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5460728e12e748eeab5f345b25177f62', 'request_id': 'req-54bff86b-8747-42bf-9cf2-d2ec5d88876c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:24.809112 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-54bff86b-8747-42bf-9cf2-d2ec5d88876c admin admin] The token cache is being invalidate because project 5460728e12e748eeab5f345b25177f62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:56:24.809405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-54bff86b-8747-42bf-9cf2-d2ec5d88876c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:24.810254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-54bff86b-8747-42bf-9cf2-d2ec5d88876c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:24.823516 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1268/2564] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:56:24 2022] DELETE /identity/v3/projects/5460728e12e748eeab5f345b25177f62 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:24.823650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00af6d-e043-4d24-af67-d1202d42e908 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:24.823650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00af6d-e043-4d24-af67-d1202d42e908 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:24.823650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00af6d-e043-4d24-af67-d1202d42e908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:24.857437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ee00af6d-e043-4d24-af67-d1202d42e908 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:24.857437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ee00af6d-e043-4d24-af67-d1202d42e908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:24.934288 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:24.967511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1297/2565] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:24.975425 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ade953-e141-4ad6-a504-babef552e317 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:24.975715 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ade953-e141-4ad6-a504-babef552e317 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:24.975990 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ade953-e141-4ad6-a504-babef552e317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:25.000560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-08ade953-e141-4ad6-a504-babef552e317 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:25.001824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-08ade953-e141-4ad6-a504-babef552e317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:25.015758 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1269/2566] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:25.019409 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00af6d-e043-4d24-af67-d1202d42e908 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:25.030628 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00af6d-e043-4d24-af67-d1202d42e908 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:25.035741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af0294-abdd-4f8a-b6da-be45ba5f49c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:25.036523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af0294-abdd-4f8a-b6da-be45ba5f49c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:25.036873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af0294-abdd-4f8a-b6da-be45ba5f49c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:25.037267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af0294-abdd-4f8a-b6da-be45ba5f49c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:25.038062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af0294-abdd-4f8a-b6da-be45ba5f49c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:56:25.038441 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af0294-abdd-4f8a-b6da-be45ba5f49c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:25.040162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af0294-abdd-4f8a-b6da-be45ba5f49c2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:25.059965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1298/2567] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:56:25.071459 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ade953-e141-4ad6-a504-babef552e317 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:25.086937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ade953-e141-4ad6-a504-babef552e317 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:25.095048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd6dd3-89b2-438f-bdf2-3ff9c6255000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:25.101073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd6dd3-89b2-438f-bdf2-3ff9c6255000 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:25.101460 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd6dd3-89b2-438f-bdf2-3ff9c6255000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:25.101698 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd6dd3-89b2-438f-bdf2-3ff9c6255000 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:25.102361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd6dd3-89b2-438f-bdf2-3ff9c6255000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:25.104414 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd6dd3-89b2-438f-bdf2-3ff9c6255000 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:25.109219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:25.215444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1270/2568] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:56:25 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:25.220287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af0294-abdd-4f8a-b6da-be45ba5f49c2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:25.237997 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af0294-abdd-4f8a-b6da-be45ba5f49c2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:25.327096 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fabb5-a1c9-4c1f-9f70-45a8ddb3fbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:25.327877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fabb5-a1c9-4c1f-9f70-45a8ddb3fbfc admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:25.328262 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fabb5-a1c9-4c1f-9f70-45a8ddb3fbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:25.328607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fabb5-a1c9-4c1f-9f70-45a8ddb3fbfc admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:25.329305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fabb5-a1c9-4c1f-9f70-45a8ddb3fbfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:25.331400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fabb5-a1c9-4c1f-9f70-45a8ddb3fbfc admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:25.421119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1299/2569] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:56:25 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:25.423420 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd6dd3-89b2-438f-bdf2-3ff9c6255000 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:25.439655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd6dd3-89b2-438f-bdf2-3ff9c6255000 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:25.445236 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b626b4-cf09-4ab5-ac2f-b6ec6b7ecb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:25.446110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b626b4-cf09-4ab5-ac2f-b6ec6b7ecb1d admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:25.446558 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b626b4-cf09-4ab5-ac2f-b6ec6b7ecb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:25.446989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b626b4-cf09-4ab5-ac2f-b6ec6b7ecb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:25.447810 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b626b4-cf09-4ab5-ac2f-b6ec6b7ecb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:25.449629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b626b4-cf09-4ab5-ac2f-b6ec6b7ecb1d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:25.465598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1271/2570] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:25.469808 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fabb5-a1c9-4c1f-9f70-45a8ddb3fbfc admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:25.481593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fabb5-a1c9-4c1f-9f70-45a8ddb3fbfc admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:25.488433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359299d7-120f-4ef7-a70b-a67e13ddbd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:25.488688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359299d7-120f-4ef7-a70b-a67e13ddbd58 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:25.488931 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359299d7-120f-4ef7-a70b-a67e13ddbd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:25.489216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359299d7-120f-4ef7-a70b-a67e13ddbd58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:25.489950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359299d7-120f-4ef7-a70b-a67e13ddbd58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:25.491813 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359299d7-120f-4ef7-a70b-a67e13ddbd58 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:25.505994 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1300/2571] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:25.510011 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b626b4-cf09-4ab5-ac2f-b6ec6b7ecb1d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:25.520902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b626b4-cf09-4ab5-ac2f-b6ec6b7ecb1d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:25.526066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314bffe-bc6b-41c4-9619-ef59106d6786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:25.526739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314bffe-bc6b-41c4-9619-ef59106d6786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:25.527151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314bffe-bc6b-41c4-9619-ef59106d6786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:25.527431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314bffe-bc6b-41c4-9619-ef59106d6786 admin admin] PATH_INFO: `/v3/projects/d28aa4e2ac774574b4f72ea8cb08334b/users/6ed400cb07b84176b7bc0c5b56284695/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:25.550359 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314bffe-bc6b-41c4-9619-ef59106d6786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d28aa4e2ac774574b4f72ea8cb08334b, user_id=6ed400cb07b84176b7bc0c5b56284695, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:25.553683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314bffe-bc6b-41c4-9619-ef59106d6786 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:25.670851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1272/2572] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:25 2022] PUT /identity/v3/projects/d28aa4e2ac774574b4f72ea8cb08334b/users/6ed400cb07b84176b7bc0c5b56284695/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:25.683418 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810a16b-cbca-48d8-829b-b8ad57a088cc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:25.683715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810a16b-cbca-48d8-829b-b8ad57a088cc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:25.683992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810a16b-cbca-48d8-829b-b8ad57a088cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:25.732610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-b810a16b-cbca-48d8-829b-b8ad57a088cc None None] MFA Rules not processed for user `6ed400cb07b84176b7bc0c5b56284695`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:25.735232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b810a16b-cbca-48d8-829b-b8ad57a088cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:25.831967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1301/2573] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:25.836303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314bffe-bc6b-41c4-9619-ef59106d6786 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:25.925166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314bffe-bc6b-41c4-9619-ef59106d6786 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:25.938959 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef5fc4-a98b-4944-80bb-f6b0060d7ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:25.939736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef5fc4-a98b-4944-80bb-f6b0060d7ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:25.941329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef5fc4-a98b-4944-80bb-f6b0060d7ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:25.942149 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef5fc4-a98b-4944-80bb-f6b0060d7ac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:25.947724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef5fc4-a98b-4944-80bb-f6b0060d7ac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:25.950122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef5fc4-a98b-4944-80bb-f6b0060d7ac0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:26.072978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1273/2574] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:56:25 2022] POST /identity/v3/projects => generated 371 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:26.077470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810a16b-cbca-48d8-829b-b8ad57a088cc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:26.095429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810a16b-cbca-48d8-829b-b8ad57a088cc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:26.136867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:26.180492 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc7726-7c56-4590-a60e-fc6d1d970740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:26.184484 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc7726-7c56-4590-a60e-fc6d1d970740 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:26.184767 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc7726-7c56-4590-a60e-fc6d1d970740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:26.185037 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc7726-7c56-4590-a60e-fc6d1d970740 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:26.185974 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc7726-7c56-4590-a60e-fc6d1d970740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:26.188208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc7726-7c56-4590-a60e-fc6d1d970740 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:26.296529 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1302/2575] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:56:26 2022] POST /identity/v3/users => generated 292 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:26.314019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef5fc4-a98b-4944-80bb-f6b0060d7ac0 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:26.327650 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef5fc4-a98b-4944-80bb-f6b0060d7ac0 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:26.334736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f225217-abbb-4f63-98ec-39569ad443b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:26.335779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f225217-abbb-4f63-98ec-39569ad443b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:26.336497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f225217-abbb-4f63-98ec-39569ad443b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:26.336946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f225217-abbb-4f63-98ec-39569ad443b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:26.339163 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f225217-abbb-4f63-98ec-39569ad443b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:26.343947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f225217-abbb-4f63-98ec-39569ad443b9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:26.359971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1274/2576] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:26.366477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc7726-7c56-4590-a60e-fc6d1d970740 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:26.376999 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc7726-7c56-4590-a60e-fc6d1d970740 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:26.395861 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0e974-8f3b-4939-b6bf-119b672b8d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:26.396674 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0e974-8f3b-4939-b6bf-119b672b8d80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:26.397021 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0e974-8f3b-4939-b6bf-119b672b8d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:26.397400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0e974-8f3b-4939-b6bf-119b672b8d80 admin admin] PATH_INFO: `/v3/projects/5c3a94f3e1d14c58a406f9316b0f3af8/users/2e713a1fd64244b9a772dbc208c995a2/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:26.414654 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0e974-8f3b-4939-b6bf-119b672b8d80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c3a94f3e1d14c58a406f9316b0f3af8, user_id=2e713a1fd64244b9a772dbc208c995a2, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:26.417912 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0e974-8f3b-4939-b6bf-119b672b8d80 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:26.729476 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1303/2577] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:26 2022] PUT /identity/v3/projects/5c3a94f3e1d14c58a406f9316b0f3af8/users/2e713a1fd64244b9a772dbc208c995a2/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:26.734287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f225217-abbb-4f63-98ec-39569ad443b9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:26.748051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f225217-abbb-4f63-98ec-39569ad443b9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:26.836302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e91e1-5654-4ec1-8d93-fb9d1f75d30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:26.836963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e91e1-5654-4ec1-8d93-fb9d1f75d30f admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:26.837244 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e91e1-5654-4ec1-8d93-fb9d1f75d30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:26.837538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e91e1-5654-4ec1-8d93-fb9d1f75d30f admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:26.838197 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e91e1-5654-4ec1-8d93-fb9d1f75d30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:26.839807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e91e1-5654-4ec1-8d93-fb9d1f75d30f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:26.851662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1275/2578] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:26.865621 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0e974-8f3b-4939-b6bf-119b672b8d80 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:26.874216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0e974-8f3b-4939-b6bf-119b672b8d80 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:26.887968 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a207e79-e011-45b1-a3f0-b59e143c629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:26.888723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a207e79-e011-45b1-a3f0-b59e143c629c admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:26.889098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a207e79-e011-45b1-a3f0-b59e143c629c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:26.889309 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a207e79-e011-45b1-a3f0-b59e143c629c admin admin] PATH_INFO: `/v3/projects/5c3a94f3e1d14c58a406f9316b0f3af8/users/2e713a1fd64244b9a772dbc208c995a2/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:26.891492 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a207e79-e011-45b1-a3f0-b59e143c629c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c3a94f3e1d14c58a406f9316b0f3af8, user_id=2e713a1fd64244b9a772dbc208c995a2, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:26.894759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a207e79-e011-45b1-a3f0-b59e143c629c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:26.907305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1304/2579] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:26 2022] PUT /identity/v3/projects/5c3a94f3e1d14c58a406f9316b0f3af8/users/2e713a1fd64244b9a772dbc208c995a2/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:56:26.921665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe84c14-5218-490f-a78b-f18c2aa8e575 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:26.922051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe84c14-5218-490f-a78b-f18c2aa8e575 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:26.922334 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe84c14-5218-490f-a78b-f18c2aa8e575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:26.961435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-cfe84c14-5218-490f-a78b-f18c2aa8e575 None None] MFA Rules not processed for user `2e713a1fd64244b9a772dbc208c995a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:26.963458 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cfe84c14-5218-490f-a78b-f18c2aa8e575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:26.975739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:27.081575 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1276/2580] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:27.106581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a207e79-e011-45b1-a3f0-b59e143c629c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:27.109767 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4a207e79-e011-45b1-a3f0-b59e143c629c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:27.224130 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a207e79-e011-45b1-a3f0-b59e143c629c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:27.233628 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d368a1-7f50-4fb3-8b65-4de686cf7d0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:27.234702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d368a1-7f50-4fb3-8b65-4de686cf7d0e service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:27.235101 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d368a1-7f50-4fb3-8b65-4de686cf7d0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:27.235696 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d368a1-7f50-4fb3-8b65-4de686cf7d0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:27.247499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d368a1-7f50-4fb3-8b65-4de686cf7d0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:27.250326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d368a1-7f50-4fb3-8b65-4de686cf7d0e service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:27.279206 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1305/2581] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:27.493303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe84c14-5218-490f-a78b-f18c2aa8e575 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:27.507169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe84c14-5218-490f-a78b-f18c2aa8e575 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:27.510855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d5108-2f0e-43da-9878-7f66baad2ddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:27.511431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d5108-2f0e-43da-9878-7f66baad2ddc service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:27.511712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d5108-2f0e-43da-9878-7f66baad2ddc service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:27.511980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d5108-2f0e-43da-9878-7f66baad2ddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:27.571614 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d5108-2f0e-43da-9878-7f66baad2ddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:27.573051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d5108-2f0e-43da-9878-7f66baad2ddc service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:27.583204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1277/2582] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:29.139118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d368a1-7f50-4fb3-8b65-4de686cf7d0e service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:29.140562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-74d368a1-7f50-4fb3-8b65-4de686cf7d0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:29.198882 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d368a1-7f50-4fb3-8b65-4de686cf7d0e service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:29.202500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a510478-1e29-43f2-a96c-4ed3d6a876d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:29.203099 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a510478-1e29-43f2-a96c-4ed3d6a876d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:29.203375 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a510478-1e29-43f2-a96c-4ed3d6a876d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:29.203645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a510478-1e29-43f2-a96c-4ed3d6a876d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:29.205647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5a510478-1e29-43f2-a96c-4ed3d6a876d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:29.253502 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:29.280209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a510478-1e29-43f2-a96c-4ed3d6a876d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:29.281707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a510478-1e29-43f2-a96c-4ed3d6a876d5 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:29.292534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1306/2583] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:34.591850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d5108-2f0e-43da-9878-7f66baad2ddc service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:34.594568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-021d5108-2f0e-43da-9878-7f66baad2ddc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:34.608422 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d5108-2f0e-43da-9878-7f66baad2ddc service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:34.614963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b3d86-a386-4415-883e-d88604ac0c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:34.615721 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b3d86-a386-4415-883e-d88604ac0c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:34.616105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b3d86-a386-4415-883e-d88604ac0c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:34.616304 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b3d86-a386-4415-883e-d88604ac0c9e admin admin] PATH_INFO: `/v3/users/00491255f4b44125b6dc9f816c9bbb58` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:34.617470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b3d86-a386-4415-883e-d88604ac0c9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=00491255f4b44125b6dc9f816c9bbb58)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:34.619602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b3d86-a386-4415-883e-d88604ac0c9e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:34.819471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a28b3d86-a386-4415-883e-d88604ac0c9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00491255f4b44125b6dc9f816c9bbb58'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:34.898433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a28b3d86-a386-4415-883e-d88604ac0c9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00491255f4b44125b6dc9f816c9bbb58'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:34.899310 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a28b3d86-a386-4415-883e-d88604ac0c9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00491255f4b44125b6dc9f816c9bbb58'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:34.909040 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a28b3d86-a386-4415-883e-d88604ac0c9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00491255f4b44125b6dc9f816c9bbb58'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:34.926032 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-a28b3d86-a386-4415-883e-d88604ac0c9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00491255f4b44125b6dc9f816c9bbb58'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:34.933791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1278/2584] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:56:34 2022] DELETE /identity/v3/users/00491255f4b44125b6dc9f816c9bbb58 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:56:35.241267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a510478-1e29-43f2-a96c-4ed3d6a876d5 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:35.260670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a510478-1e29-43f2-a96c-4ed3d6a876d5 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:35.336037 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:35.336658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:35.337075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:35.337351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a admin admin] PATH_INFO: `/v3/projects/4a745db383cb452187f7c82b9977c09b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:35.338618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a745db383cb452187f7c82b9977c09b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:35.340516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:35.490806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a745db383cb452187f7c82b9977c09b', 'request_id': 'req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:35.497138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a745db383cb452187f7c82b9977c09b', 'request_id': 'req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:35.504331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a745db383cb452187f7c82b9977c09b', 'request_id': 'req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:35.591240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a admin admin] The token cache is being invalidate because project 4a745db383cb452187f7c82b9977c09b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:56:35.591741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:35.592529 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-ee4ece4d-91d0-4cc6-b5b4-46927ffa378a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:35.595434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1307/2585] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:56:35 2022] DELETE /identity/v3/projects/4a745db383cb452187f7c82b9977c09b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:35.605325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95c673-cfbe-490b-b39b-2c871dc6415f None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:35.605849 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95c673-cfbe-490b-b39b-2c871dc6415f None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:35.606103 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95c673-cfbe-490b-b39b-2c871dc6415f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:35.626345 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-cc95c673-cfbe-490b-b39b-2c871dc6415f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:35.627527 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-cc95c673-cfbe-490b-b39b-2c871dc6415f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:35.637411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:35.690631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1279/2586] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:56:35.698775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c189b-4016-4565-a998-a043a2e4bcb2 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:35.699093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c189b-4016-4565-a998-a043a2e4bcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:35.699378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c189b-4016-4565-a998-a043a2e4bcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:35.721169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-661c189b-4016-4565-a998-a043a2e4bcb2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:35.723246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-661c189b-4016-4565-a998-a043a2e4bcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:35.731463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1308/2587] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:35.735862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95c673-cfbe-490b-b39b-2c871dc6415f None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:35.746272 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95c673-cfbe-490b-b39b-2c871dc6415f None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:35.751306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64185635-4ee3-4863-90f9-444f5468bb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:35.751915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64185635-4ee3-4863-90f9-444f5468bb13 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:35.752195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64185635-4ee3-4863-90f9-444f5468bb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:35.752466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64185635-4ee3-4863-90f9-444f5468bb13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:35.753146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64185635-4ee3-4863-90f9-444f5468bb13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:56:35.753471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64185635-4ee3-4863-90f9-444f5468bb13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:35.755421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64185635-4ee3-4863-90f9-444f5468bb13 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:35.771628 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1280/2588] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:56:35.772729 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c189b-4016-4565-a998-a043a2e4bcb2 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:35.782707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c189b-4016-4565-a998-a043a2e4bcb2 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:35.788029 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c78b95-fb66-41d7-8234-4deabb9a1265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:35.788646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c78b95-fb66-41d7-8234-4deabb9a1265 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:35.788921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c78b95-fb66-41d7-8234-4deabb9a1265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:35.789214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c78b95-fb66-41d7-8234-4deabb9a1265 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:35.789866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c78b95-fb66-41d7-8234-4deabb9a1265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:35.791780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c78b95-fb66-41d7-8234-4deabb9a1265 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:35.914738 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1309/2589] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:56:35 2022] POST /identity/v3/projects => generated 405 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:35.915935 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64185635-4ee3-4863-90f9-444f5468bb13 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:35.926240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64185635-4ee3-4863-90f9-444f5468bb13 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:35.991997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b91636-263c-479a-813e-8b6e25af6dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:35.992612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b91636-263c-479a-813e-8b6e25af6dad admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:35.992901 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b91636-263c-479a-813e-8b6e25af6dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:35.993169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b91636-263c-479a-813e-8b6e25af6dad admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:35.993828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b91636-263c-479a-813e-8b6e25af6dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:35.995732 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b91636-263c-479a-813e-8b6e25af6dad admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:36.076499 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1281/2590] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:56:35 2022] POST /identity/v3/users => generated 303 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:56:36.079630 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c78b95-fb66-41d7-8234-4deabb9a1265 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:36.090936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c78b95-fb66-41d7-8234-4deabb9a1265 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:36.095916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1a13a-b8b2-40b7-88ec-2b9f94f3e21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:36.096703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1a13a-b8b2-40b7-88ec-2b9f94f3e21d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:36.097539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1a13a-b8b2-40b7-88ec-2b9f94f3e21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:36.097822 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1a13a-b8b2-40b7-88ec-2b9f94f3e21d admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:36.098462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1a13a-b8b2-40b7-88ec-2b9f94f3e21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:36.100056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1a13a-b8b2-40b7-88ec-2b9f94f3e21d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:36.110428 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1310/2591] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:36.114683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b91636-263c-479a-813e-8b6e25af6dad admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:36.124937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b91636-263c-479a-813e-8b6e25af6dad admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:36.130289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402e72f-86e0-49f2-85dc-b6e493b99c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:36.130954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402e72f-86e0-49f2-85dc-b6e493b99c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:36.131215 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402e72f-86e0-49f2-85dc-b6e493b99c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:36.131492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402e72f-86e0-49f2-85dc-b6e493b99c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:36.132133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402e72f-86e0-49f2-85dc-b6e493b99c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:36.133749 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402e72f-86e0-49f2-85dc-b6e493b99c0f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:36.144815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1282/2592] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:36.149026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1a13a-b8b2-40b7-88ec-2b9f94f3e21d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:36.159266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1a13a-b8b2-40b7-88ec-2b9f94f3e21d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:36.164169 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309cd90-0cf5-4cb6-91d4-538e45ebacf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:36.164827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309cd90-0cf5-4cb6-91d4-538e45ebacf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:36.165149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309cd90-0cf5-4cb6-91d4-538e45ebacf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:36.165431 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309cd90-0cf5-4cb6-91d4-538e45ebacf7 admin admin] PATH_INFO: `/v3/projects/2f4647cc210840e382b63e07d98c900d/users/9b8c756c344043b5976a015051cb4407/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:36.180361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309cd90-0cf5-4cb6-91d4-538e45ebacf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f4647cc210840e382b63e07d98c900d, user_id=9b8c756c344043b5976a015051cb4407, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:36.183103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309cd90-0cf5-4cb6-91d4-538e45ebacf7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:36.235723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1311/2593] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:36 2022] PUT /identity/v3/projects/2f4647cc210840e382b63e07d98c900d/users/9b8c756c344043b5976a015051cb4407/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:36.240780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ee30e-adad-4c15-9c10-a9c4f40a94e2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:36.241066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ee30e-adad-4c15-9c10-a9c4f40a94e2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:36.241338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ee30e-adad-4c15-9c10-a9c4f40a94e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:36.260601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-da0ee30e-adad-4c15-9c10-a9c4f40a94e2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:36.261861 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-da0ee30e-adad-4c15-9c10-a9c4f40a94e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:36.321111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1283/2594] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:36.330722 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309cd90-0cf5-4cb6-91d4-538e45ebacf7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:36.333000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3309cd90-0cf5-4cb6-91d4-538e45ebacf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:36.339440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:36.388818 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309cd90-0cf5-4cb6-91d4-538e45ebacf7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:36.392898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcf2ca-ea67-43d4-a88a-1584912ace5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:36.394095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcf2ca-ea67-43d4-a88a-1584912ace5a service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:36.394391 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcf2ca-ea67-43d4-a88a-1584912ace5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:36.394657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcf2ca-ea67-43d4-a88a-1584912ace5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:36.403652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcf2ca-ea67-43d4-a88a-1584912ace5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:36.405072 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcf2ca-ea67-43d4-a88a-1584912ace5a service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:36.414731 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1312/2595] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:37.920884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7458e4a-f2a7-450c-972c-4a1477c0bfe6 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:37.921450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7458e4a-f2a7-450c-972c-4a1477c0bfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:37.921450 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7458e4a-f2a7-450c-972c-4a1477c0bfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:37.954007 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-c7458e4a-f2a7-450c-972c-4a1477c0bfe6 None None] MFA Rules not processed for user `9b8c756c344043b5976a015051cb4407`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:37.955119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-c7458e4a-f2a7-450c-972c-4a1477c0bfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:37.965683 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:38.029205 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1284/2596] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:37 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:56:38.033967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcf2ca-ea67-43d4-a88a-1584912ace5a service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:38.044378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcf2ca-ea67-43d4-a88a-1584912ace5a service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:38.049182 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88c848-89f6-4ec5-b189-c00ef265d8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:38.049828 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88c848-89f6-4ec5-b189-c00ef265d8f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:38.050112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88c848-89f6-4ec5-b189-c00ef265d8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:38.050384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88c848-89f6-4ec5-b189-c00ef265d8f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:38.051053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88c848-89f6-4ec5-b189-c00ef265d8f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:38.052975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88c848-89f6-4ec5-b189-c00ef265d8f3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:38.142666 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1313/2597] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:56:38 2022] POST /identity/v3/projects => generated 403 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:38.144406 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7458e4a-f2a7-450c-972c-4a1477c0bfe6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:38.156426 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7458e4a-f2a7-450c-972c-4a1477c0bfe6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:38.217025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac0d17-7ad8-4596-8f84-fe7caf8d1865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:38.217682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac0d17-7ad8-4596-8f84-fe7caf8d1865 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:38.217968 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac0d17-7ad8-4596-8f84-fe7caf8d1865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:38.218242 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac0d17-7ad8-4596-8f84-fe7caf8d1865 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:38.218875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac0d17-7ad8-4596-8f84-fe7caf8d1865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:38.220864 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac0d17-7ad8-4596-8f84-fe7caf8d1865 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:38.335846 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1285/2598] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:56:38 2022] POST /identity/v3/users => generated 308 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:38.338617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88c848-89f6-4ec5-b189-c00ef265d8f3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:38.349250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88c848-89f6-4ec5-b189-c00ef265d8f3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:38.353785 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a48305-df62-422d-bc05-cc35a242d849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:38.354378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a48305-df62-422d-bc05-cc35a242d849 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:38.354657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a48305-df62-422d-bc05-cc35a242d849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:38.354927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a48305-df62-422d-bc05-cc35a242d849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:38.355562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a48305-df62-422d-bc05-cc35a242d849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:38.357147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a48305-df62-422d-bc05-cc35a242d849 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:38.368642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1314/2599] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:38.379411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac0d17-7ad8-4596-8f84-fe7caf8d1865 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:38.389764 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac0d17-7ad8-4596-8f84-fe7caf8d1865 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:38.395028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f570f-43af-4fb5-9cbb-3d1d078bcb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:38.395678 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f570f-43af-4fb5-9cbb-3d1d078bcb8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:38.395963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f570f-43af-4fb5-9cbb-3d1d078bcb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:38.396240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f570f-43af-4fb5-9cbb-3d1d078bcb8c admin admin] PATH_INFO: `/v3/projects/82ea84a4ce45417f8b44c4c729c2d172/users/8b724d673a5a4109af344088b3afa23a/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:38.412333 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f570f-43af-4fb5-9cbb-3d1d078bcb8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ea84a4ce45417f8b44c4c729c2d172, user_id=8b724d673a5a4109af344088b3afa23a, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:38.415004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f570f-43af-4fb5-9cbb-3d1d078bcb8c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:38.487154 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1286/2600] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:38 2022] PUT /identity/v3/projects/82ea84a4ce45417f8b44c4c729c2d172/users/8b724d673a5a4109af344088b3afa23a/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:56:38.491280 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a48305-df62-422d-bc05-cc35a242d849 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:38.501205 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a48305-df62-422d-bc05-cc35a242d849 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:38.563961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f94fb-a12c-4c87-99f9-3975d22be05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:38.564620 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f94fb-a12c-4c87-99f9-3975d22be05e admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:38.565035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f94fb-a12c-4c87-99f9-3975d22be05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:38.565146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f94fb-a12c-4c87-99f9-3975d22be05e admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:38.566329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f94fb-a12c-4c87-99f9-3975d22be05e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:38.567915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f94fb-a12c-4c87-99f9-3975d22be05e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:38.577886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1315/2601] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:38.582514 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f570f-43af-4fb5-9cbb-3d1d078bcb8c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:38.592995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f570f-43af-4fb5-9cbb-3d1d078bcb8c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:38.598297 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a87f2-8378-4186-a724-2973b7f09120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:38.599156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a87f2-8378-4186-a724-2973b7f09120 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:38.599565 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a87f2-8378-4186-a724-2973b7f09120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:38.600442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a87f2-8378-4186-a724-2973b7f09120 admin admin] PATH_INFO: `/v3/projects/82ea84a4ce45417f8b44c4c729c2d172/users/8b724d673a5a4109af344088b3afa23a/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:38.602175 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a87f2-8378-4186-a724-2973b7f09120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ea84a4ce45417f8b44c4c729c2d172, user_id=8b724d673a5a4109af344088b3afa23a, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:38.605118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a87f2-8378-4186-a724-2973b7f09120 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:38.616839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1287/2602] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:38 2022] PUT /identity/v3/projects/82ea84a4ce45417f8b44c4c729c2d172/users/8b724d673a5a4109af344088b3afa23a/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:40.083797 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86277e-2f11-410e-894b-814a0e3ad9a9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:40.084355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86277e-2f11-410e-894b-814a0e3ad9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:40.084705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86277e-2f11-410e-894b-814a0e3ad9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:40.142549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-3d86277e-2f11-410e-894b-814a0e3ad9a9 None None] MFA Rules not processed for user `8b724d673a5a4109af344088b3afa23a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:40.146340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-3d86277e-2f11-410e-894b-814a0e3ad9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:40.230304 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:40.241647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1316/2603] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:40 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:41.720640 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a87f2-8378-4186-a724-2973b7f09120 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:41.796072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a87f2-8378-4186-a724-2973b7f09120 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:41.802462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:41.803200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:41.803560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:41.803838 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] PATH_INFO: `/v3/users/9b8c756c344043b5976a015051cb4407` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:41.805033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b8c756c344043b5976a015051cb4407)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:41.806949 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:41.813795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:41.998513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b8c756c344043b5976a015051cb4407'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:42.103796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b8c756c344043b5976a015051cb4407'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:42.104461 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b8c756c344043b5976a015051cb4407'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:42.114771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b8c756c344043b5976a015051cb4407'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:42.127527 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b8c756c344043b5976a015051cb4407'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:42.133836 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1288/2604] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:56:41 2022] DELETE /identity/v3/users/9b8c756c344043b5976a015051cb4407 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:42.695579 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86277e-2f11-410e-894b-814a0e3ad9a9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:42.706902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86277e-2f11-410e-894b-814a0e3ad9a9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:42.766293 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:42.766965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:42.767244 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:42.767519 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] PATH_INFO: `/v3/projects/2f4647cc210840e382b63e07d98c900d` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:42.768719 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f4647cc210840e382b63e07d98c900d)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:42.771129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:42.891686 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f4647cc210840e382b63e07d98c900d', 'request_id': 'req-7e71a1f1-0486-4eb6-9825-dd79775475f1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:42.899882 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f4647cc210840e382b63e07d98c900d', 'request_id': 'req-7e71a1f1-0486-4eb6-9825-dd79775475f1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:42.906706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f4647cc210840e382b63e07d98c900d', 'request_id': 'req-7e71a1f1-0486-4eb6-9825-dd79775475f1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:42.985626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] The token cache is being invalidate because project 2f4647cc210840e382b63e07d98c900d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:56:42.986173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:42.986917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:42.988651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1317/2605] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:56:42 2022] DELETE /identity/v3/projects/2f4647cc210840e382b63e07d98c900d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:42.992582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:42.994157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:43.010066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a94b0b-3a43-4605-9b43-11f8878eb6f2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:43.082146 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db336ac-81cc-4dc7-b0ee-b2ddbbb5bf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:43.082795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db336ac-81cc-4dc7-b0ee-b2ddbbb5bf6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:43.083081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db336ac-81cc-4dc7-b0ee-b2ddbbb5bf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:43.083361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db336ac-81cc-4dc7-b0ee-b2ddbbb5bf6b admin admin] PATH_INFO: `/v3/users/8b724d673a5a4109af344088b3afa23a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:43.084513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db336ac-81cc-4dc7-b0ee-b2ddbbb5bf6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b724d673a5a4109af344088b3afa23a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:43.086465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db336ac-81cc-4dc7-b0ee-b2ddbbb5bf6b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:43.291729 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3db336ac-81cc-4dc7-b0ee-b2ddbbb5bf6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b724d673a5a4109af344088b3afa23a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:43.379151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3db336ac-81cc-4dc7-b0ee-b2ddbbb5bf6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b724d673a5a4109af344088b3afa23a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:43.379866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3db336ac-81cc-4dc7-b0ee-b2ddbbb5bf6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b724d673a5a4109af344088b3afa23a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:43.394521 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3db336ac-81cc-4dc7-b0ee-b2ddbbb5bf6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b724d673a5a4109af344088b3afa23a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:43.418697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-3db336ac-81cc-4dc7-b0ee-b2ddbbb5bf6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b724d673a5a4109af344088b3afa23a'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:43.420101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1289/2606] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:56:42 2022] DELETE /identity/v3/users/8b724d673a5a4109af344088b3afa23a => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:43.671709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:43.673571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:43.689732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71a1f1-0486-4eb6-9825-dd79775475f1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:43.752263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65c907-f12b-4a88-a05f-e820a3d8ba98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:43.753003 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65c907-f12b-4a88-a05f-e820a3d8ba98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:43.753161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65c907-f12b-4a88-a05f-e820a3d8ba98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:43.753487 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65c907-f12b-4a88-a05f-e820a3d8ba98 admin admin] PATH_INFO: `/v3/projects/82ea84a4ce45417f8b44c4c729c2d172` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:43.754761 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65c907-f12b-4a88-a05f-e820a3d8ba98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ea84a4ce45417f8b44c4c729c2d172)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:43.756721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65c907-f12b-4a88-a05f-e820a3d8ba98 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:43.832120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:43.847704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1a65c907-f12b-4a88-a05f-e820a3d8ba98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ea84a4ce45417f8b44c4c729c2d172', 'request_id': 'req-1a65c907-f12b-4a88-a05f-e820a3d8ba98'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:43.852916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1a65c907-f12b-4a88-a05f-e820a3d8ba98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ea84a4ce45417f8b44c4c729c2d172', 'request_id': 'req-1a65c907-f12b-4a88-a05f-e820a3d8ba98'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:43.859241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1a65c907-f12b-4a88-a05f-e820a3d8ba98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ea84a4ce45417f8b44c4c729c2d172', 'request_id': 'req-1a65c907-f12b-4a88-a05f-e820a3d8ba98'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:43.983917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1a65c907-f12b-4a88-a05f-e820a3d8ba98 admin admin] The token cache is being invalidate because project 82ea84a4ce45417f8b44c4c729c2d172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:56:43.984214 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1a65c907-f12b-4a88-a05f-e820a3d8ba98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:43.984914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1a65c907-f12b-4a88-a05f-e820a3d8ba98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:43.986437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1318/2607] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:56:43 2022] DELETE /identity/v3/projects/82ea84a4ce45417f8b44c4c729c2d172 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:43.991538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90d8a9-9331-4901-9d7d-3e494fe93f65 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:43.991971 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90d8a9-9331-4901-9d7d-3e494fe93f65 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:43.992347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90d8a9-9331-4901-9d7d-3e494fe93f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:44.016389 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:44.031677 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4f90d8a9-9331-4901-9d7d-3e494fe93f65 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:44.033183 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4f90d8a9-9331-4901-9d7d-3e494fe93f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:44.108464 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1290/2608] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:44.116372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6016d1-e0c5-4ff4-8467-a8b153b13fe5 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:44.116669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6016d1-e0c5-4ff4-8467-a8b153b13fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:44.116949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6016d1-e0c5-4ff4-8467-a8b153b13fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:44.137178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4e6016d1-e0c5-4ff4-8467-a8b153b13fe5 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:44.138421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4e6016d1-e0c5-4ff4-8467-a8b153b13fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:44.147497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1319/2609] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:44.151940 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90d8a9-9331-4901-9d7d-3e494fe93f65 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:44.163014 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90d8a9-9331-4901-9d7d-3e494fe93f65 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:44.167702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dcfce-5396-4694-8b70-875a8e3fef35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:44.168344 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dcfce-5396-4694-8b70-875a8e3fef35 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:44.168617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dcfce-5396-4694-8b70-875a8e3fef35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:44.168882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dcfce-5396-4694-8b70-875a8e3fef35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:44.169541 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dcfce-5396-4694-8b70-875a8e3fef35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:56:44.169857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dcfce-5396-4694-8b70-875a8e3fef35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:44.171472 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dcfce-5396-4694-8b70-875a8e3fef35 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:44.185688 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1291/2610] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:56:44.190649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6016d1-e0c5-4ff4-8467-a8b153b13fe5 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:44.199940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6016d1-e0c5-4ff4-8467-a8b153b13fe5 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:44.204601 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca140ef-f8d4-41e1-9bbc-cba45ece4503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:44.205196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca140ef-f8d4-41e1-9bbc-cba45ece4503 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:44.205470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca140ef-f8d4-41e1-9bbc-cba45ece4503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:44.205769 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca140ef-f8d4-41e1-9bbc-cba45ece4503 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:44.209704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca140ef-f8d4-41e1-9bbc-cba45ece4503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:44.209704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca140ef-f8d4-41e1-9bbc-cba45ece4503 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:44.294367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1320/2611] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:56:44 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:56:44.298663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dcfce-5396-4694-8b70-875a8e3fef35 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:44.308710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dcfce-5396-4694-8b70-875a8e3fef35 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:44.369361 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6e996-b24c-4646-b52e-12f6e7ad82ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:44.369994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6e996-b24c-4646-b52e-12f6e7ad82ad admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:44.370271 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6e996-b24c-4646-b52e-12f6e7ad82ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:44.370537 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6e996-b24c-4646-b52e-12f6e7ad82ad admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:44.371170 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6e996-b24c-4646-b52e-12f6e7ad82ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:44.382830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6e996-b24c-4646-b52e-12f6e7ad82ad admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:44.493302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1292/2612] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:56:44 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:44.497638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca140ef-f8d4-41e1-9bbc-cba45ece4503 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:44.509036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca140ef-f8d4-41e1-9bbc-cba45ece4503 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:44.513570 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027aa6a-a10d-4a9d-9898-2ca541d27c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:44.514165 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027aa6a-a10d-4a9d-9898-2ca541d27c6f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:44.514438 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027aa6a-a10d-4a9d-9898-2ca541d27c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:44.514709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027aa6a-a10d-4a9d-9898-2ca541d27c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:44.515347 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027aa6a-a10d-4a9d-9898-2ca541d27c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:44.517094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027aa6a-a10d-4a9d-9898-2ca541d27c6f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:44.527952 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1321/2613] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:44.532023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6e996-b24c-4646-b52e-12f6e7ad82ad admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:44.543849 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6e996-b24c-4646-b52e-12f6e7ad82ad admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:44.548471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8bf78-336f-4494-acfb-56ec2d9fc3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:44.549132 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8bf78-336f-4494-acfb-56ec2d9fc3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:44.549351 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8bf78-336f-4494-acfb-56ec2d9fc3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:44.549663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8bf78-336f-4494-acfb-56ec2d9fc3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:44.550313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8bf78-336f-4494-acfb-56ec2d9fc3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:44.551900 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8bf78-336f-4494-acfb-56ec2d9fc3f2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:44.564143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1293/2614] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:44.568983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027aa6a-a10d-4a9d-9898-2ca541d27c6f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:44.579624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027aa6a-a10d-4a9d-9898-2ca541d27c6f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:44.584907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9633dee-0a32-4fad-a238-fe66e1845b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:44.585644 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9633dee-0a32-4fad-a238-fe66e1845b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:44.585918 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9633dee-0a32-4fad-a238-fe66e1845b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:44.586194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9633dee-0a32-4fad-a238-fe66e1845b9b admin admin] PATH_INFO: `/v3/projects/d3275b9226ce4340af8c09a94e6a7974/users/2b397880f9eb4318b38a7aae74e14b6f/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:44.601735 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9633dee-0a32-4fad-a238-fe66e1845b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3275b9226ce4340af8c09a94e6a7974, user_id=2b397880f9eb4318b38a7aae74e14b6f, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:44.604392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9633dee-0a32-4fad-a238-fe66e1845b9b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:44.675866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1322/2615] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:44 2022] PUT /identity/v3/projects/d3275b9226ce4340af8c09a94e6a7974/users/2b397880f9eb4318b38a7aae74e14b6f/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:44.687792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b88a3-bd51-4642-af8a-cf533e5f78ba None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:44.688090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b88a3-bd51-4642-af8a-cf533e5f78ba None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:44.688410 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b88a3-bd51-4642-af8a-cf533e5f78ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:44.717142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4b7b88a3-bd51-4642-af8a-cf533e5f78ba None None] MFA Rules not processed for user `2b397880f9eb4318b38a7aae74e14b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:44.718550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4b7b88a3-bd51-4642-af8a-cf533e5f78ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:44.780898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:44.787156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1294/2616] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:44.791852 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9633dee-0a32-4fad-a238-fe66e1845b9b admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:44.860062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9633dee-0a32-4fad-a238-fe66e1845b9b admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:44.865413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6a0cb-fa80-468f-88f9-9378b76c361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:44.866053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6a0cb-fa80-468f-88f9-9378b76c361a admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:44.866335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6a0cb-fa80-468f-88f9-9378b76c361a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:44.866610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6a0cb-fa80-468f-88f9-9378b76c361a admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:44.867250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6a0cb-fa80-468f-88f9-9378b76c361a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:44.869316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6a0cb-fa80-468f-88f9-9378b76c361a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:44.979506 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1323/2617] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:56:44 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:44.983425 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b88a3-bd51-4642-af8a-cf533e5f78ba None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:44.995702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b88a3-bd51-4642-af8a-cf533e5f78ba None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:45.068375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff2bde-58b2-4047-849f-583cec3c8673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:45.069004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff2bde-58b2-4047-849f-583cec3c8673 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:45.069282 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff2bde-58b2-4047-849f-583cec3c8673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:45.069581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff2bde-58b2-4047-849f-583cec3c8673 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:45.070232 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff2bde-58b2-4047-849f-583cec3c8673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:45.072178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff2bde-58b2-4047-849f-583cec3c8673 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:45.170691 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1295/2618] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:56:44 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:45.179110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6a0cb-fa80-468f-88f9-9378b76c361a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:45.191298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6a0cb-fa80-468f-88f9-9378b76c361a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:45.196168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628f5bc-2832-47c6-bcb9-173a46651ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:45.196772 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628f5bc-2832-47c6-bcb9-173a46651ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:45.197053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628f5bc-2832-47c6-bcb9-173a46651ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:45.197326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628f5bc-2832-47c6-bcb9-173a46651ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:45.198004 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628f5bc-2832-47c6-bcb9-173a46651ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:45.199600 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628f5bc-2832-47c6-bcb9-173a46651ab6 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:45.215174 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1324/2619] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:45.216335 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff2bde-58b2-4047-849f-583cec3c8673 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:45.276035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff2bde-58b2-4047-849f-583cec3c8673 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:45.280706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba2b1d-c9dc-4003-a0cf-86173f55ad8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:45.281374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba2b1d-c9dc-4003-a0cf-86173f55ad8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:45.281699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba2b1d-c9dc-4003-a0cf-86173f55ad8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:45.281978 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba2b1d-c9dc-4003-a0cf-86173f55ad8f admin admin] PATH_INFO: `/v3/projects/a83fa7f0e9e848db8e5965102f396160/users/69577cbcd89840718a61289221898c6d/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:45.301329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba2b1d-c9dc-4003-a0cf-86173f55ad8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a83fa7f0e9e848db8e5965102f396160, user_id=69577cbcd89840718a61289221898c6d, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:45.304378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba2b1d-c9dc-4003-a0cf-86173f55ad8f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:45.408754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1296/2620] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:45 2022] PUT /identity/v3/projects/a83fa7f0e9e848db8e5965102f396160/users/69577cbcd89840718a61289221898c6d/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:45.417759 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628f5bc-2832-47c6-bcb9-173a46651ab6 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:45.451742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:45.465043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628f5bc-2832-47c6-bcb9-173a46651ab6 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:45.527569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc934e-01a3-43f3-b55d-ea6230693820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:45.528195 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc934e-01a3-43f3-b55d-ea6230693820 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:45.528480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc934e-01a3-43f3-b55d-ea6230693820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:45.528753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc934e-01a3-43f3-b55d-ea6230693820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:45.529399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc934e-01a3-43f3-b55d-ea6230693820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:45.531164 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc934e-01a3-43f3-b55d-ea6230693820 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:45.541743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1325/2621] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:45.545845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba2b1d-c9dc-4003-a0cf-86173f55ad8f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:45.557826 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba2b1d-c9dc-4003-a0cf-86173f55ad8f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:45.563037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c0273-a9ec-415e-880e-c13c5f9087d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:45.563813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c0273-a9ec-415e-880e-c13c5f9087d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:45.564098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c0273-a9ec-415e-880e-c13c5f9087d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:45.564374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c0273-a9ec-415e-880e-c13c5f9087d7 admin admin] PATH_INFO: `/v3/projects/a83fa7f0e9e848db8e5965102f396160/users/69577cbcd89840718a61289221898c6d/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:45.566411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c0273-a9ec-415e-880e-c13c5f9087d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a83fa7f0e9e848db8e5965102f396160, user_id=69577cbcd89840718a61289221898c6d, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:45.569069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c0273-a9ec-415e-880e-c13c5f9087d7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:45.579681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1297/2622] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:45 2022] PUT /identity/v3/projects/a83fa7f0e9e848db8e5965102f396160/users/69577cbcd89840718a61289221898c6d/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:45.590544 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f78dea-7ac7-4737-bb84-636554b4a83f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:45.590830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f78dea-7ac7-4737-bb84-636554b4a83f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:45.591113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f78dea-7ac7-4737-bb84-636554b4a83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:45.629166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-53f78dea-7ac7-4737-bb84-636554b4a83f None None] MFA Rules not processed for user `69577cbcd89840718a61289221898c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:45.630395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-53f78dea-7ac7-4737-bb84-636554b4a83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:45.714373 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1326/2623] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:45.727506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c0273-a9ec-415e-880e-c13c5f9087d7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:45.727642 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-582c0273-a9ec-415e-880e-c13c5f9087d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:45.826038 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:45.839796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c0273-a9ec-415e-880e-c13c5f9087d7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:45.843531 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf9dfc-6d1d-45ca-8dfb-584711179e1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:45.844229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf9dfc-6d1d-45ca-8dfb-584711179e1f service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:45.844433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf9dfc-6d1d-45ca-8dfb-584711179e1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:45.844732 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf9dfc-6d1d-45ca-8dfb-584711179e1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:45.853981 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf9dfc-6d1d-45ca-8dfb-584711179e1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:45.855409 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf9dfc-6d1d-45ca-8dfb-584711179e1f service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:45.866133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1298/2624] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:45.979498 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f78dea-7ac7-4737-bb84-636554b4a83f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:45.989205 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f78dea-7ac7-4737-bb84-636554b4a83f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:45.992745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85c2f5-3a29-4e86-82d4-8ca75bec9bfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:45.993337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85c2f5-3a29-4e86-82d4-8ca75bec9bfa service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:45.993645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85c2f5-3a29-4e86-82d4-8ca75bec9bfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:45.993917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85c2f5-3a29-4e86-82d4-8ca75bec9bfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:46.049560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85c2f5-3a29-4e86-82d4-8ca75bec9bfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:46.051042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85c2f5-3a29-4e86-82d4-8ca75bec9bfa service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:46.060022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1327/2625] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:49.107691 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf9dfc-6d1d-45ca-8dfb-584711179e1f service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:49.173486 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf9dfc-6d1d-45ca-8dfb-584711179e1f service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:49.178828 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:49.179471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:49.179757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:49.180066 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] PATH_INFO: `/v3/users/2b397880f9eb4318b38a7aae74e14b6f` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:49.181190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b397880f9eb4318b38a7aae74e14b6f)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:49.183104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:49.433489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b397880f9eb4318b38a7aae74e14b6f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:49.526432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b397880f9eb4318b38a7aae74e14b6f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:49.527044 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b397880f9eb4318b38a7aae74e14b6f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:49.535211 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b397880f9eb4318b38a7aae74e14b6f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:49.548679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b397880f9eb4318b38a7aae74e14b6f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:49.556554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1299/2626] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:56:49 2022] DELETE /identity/v3/users/2b397880f9eb4318b38a7aae74e14b6f => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:49.563062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb571d8-404a-4c8b-b794-8a65448f528b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:49.567624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb571d8-404a-4c8b-b794-8a65448f528b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:49.567624 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb571d8-404a-4c8b-b794-8a65448f528b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:49.571991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:49.597337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4bb571d8-404a-4c8b-b794-8a65448f528b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:49.598830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4bb571d8-404a-4c8b-b794-8a65448f528b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:49.666259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1328/2627] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:49.677619 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:49.679731 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:49.743449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0e8fb-2f37-460e-8d26-25f33b59cdaf admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:49.747263 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264a64b-7c40-41dc-b806-9f29975e91c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:49.747871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264a64b-7c40-41dc-b806-9f29975e91c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:49.748151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264a64b-7c40-41dc-b806-9f29975e91c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:49.748423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264a64b-7c40-41dc-b806-9f29975e91c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:49.758048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264a64b-7c40-41dc-b806-9f29975e91c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:49.759582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264a64b-7c40-41dc-b806-9f29975e91c3 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:49.769411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1300/2628] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:50.185422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb571d8-404a-4c8b-b794-8a65448f528b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:50.188153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4bb571d8-404a-4c8b-b794-8a65448f528b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:50.208056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb571d8-404a-4c8b-b794-8a65448f528b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:50.215407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:50.215959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:50.216336 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:50.216599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] PATH_INFO: `/v3/projects/d3275b9226ce4340af8c09a94e6a7974` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:50.218104 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3275b9226ce4340af8c09a94e6a7974)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:50.220057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:50.310422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3275b9226ce4340af8c09a94e6a7974', 'request_id': 'req-8b81b657-ac86-458a-8dcd-f4077d61cf7c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:50.316676 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3275b9226ce4340af8c09a94e6a7974', 'request_id': 'req-8b81b657-ac86-458a-8dcd-f4077d61cf7c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:50.324387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3275b9226ce4340af8c09a94e6a7974', 'request_id': 'req-8b81b657-ac86-458a-8dcd-f4077d61cf7c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:50.394873 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] The token cache is being invalidate because project d3275b9226ce4340af8c09a94e6a7974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:56:50.395163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:50.395788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:50.398189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1329/2629] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:56:50 2022] DELETE /identity/v3/projects/d3275b9226ce4340af8c09a94e6a7974 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:50.402387 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264a64b-7c40-41dc-b806-9f29975e91c3 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:50.414570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264a64b-7c40-41dc-b806-9f29975e91c3 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:50.421500 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:50.476847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:50.477398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:50.477708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:50.477982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] PATH_INFO: `/v3/users/69577cbcd89840718a61289221898c6d` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:50.479100 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69577cbcd89840718a61289221898c6d)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:50.481062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:50.655313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69577cbcd89840718a61289221898c6d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:50.738174 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69577cbcd89840718a61289221898c6d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:50.738835 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69577cbcd89840718a61289221898c6d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:50.749805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69577cbcd89840718a61289221898c6d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:50.766044 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69577cbcd89840718a61289221898c6d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:50.778004 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1301/2630] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:56:50 2022] DELETE /identity/v3/users/69577cbcd89840718a61289221898c6d => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:51.072520 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:51.074071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:51.136749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81b657-ac86-458a-8dcd-f4077d61cf7c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:51.170092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:51.188856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e907af-768c-4c2d-a1f0-7da72eb65ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:51.189396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e907af-768c-4c2d-a1f0-7da72eb65ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:51.189733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e907af-768c-4c2d-a1f0-7da72eb65ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:51.190014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e907af-768c-4c2d-a1f0-7da72eb65ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:51.191801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-80e907af-768c-4c2d-a1f0-7da72eb65ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:51.208374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:51.211661 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:51.264295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e907af-768c-4c2d-a1f0-7da72eb65ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:51.265939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e907af-768c-4c2d-a1f0-7da72eb65ef4 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:51.278497 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1330/2631] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:51.296504 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be2d21-8991-4c90-8deb-d2537f2dcb92 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:51.301576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:51.302254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:51.302541 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:51.303512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6 admin admin] PATH_INFO: `/v3/projects/a83fa7f0e9e848db8e5965102f396160` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:51.304667 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a83fa7f0e9e848db8e5965102f396160)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:51.306617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:51.383417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:51.399209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a83fa7f0e9e848db8e5965102f396160', 'request_id': 'req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:51.408045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a83fa7f0e9e848db8e5965102f396160', 'request_id': 'req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:51.415860 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a83fa7f0e9e848db8e5965102f396160', 'request_id': 'req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:51.492815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6 admin admin] The token cache is being invalidate because project a83fa7f0e9e848db8e5965102f396160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:56:51.493320 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:51.494202 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-c7cd2c87-e291-4787-b1ef-f35b43bf42c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:56:51.495727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1302/2632] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:56:51 2022] DELETE /identity/v3/projects/a83fa7f0e9e848db8e5965102f396160 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:56:51.500751 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8447423-c65b-4cf8-b15c-6a613df3cf9f None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:51.501033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8447423-c65b-4cf8-b15c-6a613df3cf9f None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:51.501333 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8447423-c65b-4cf8-b15c-6a613df3cf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:51.558341 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e8447423-c65b-4cf8-b15c-6a613df3cf9f None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:51.559597 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e8447423-c65b-4cf8-b15c-6a613df3cf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:51.624369 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1331/2633] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:51.633150 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcc061-ac68-48a0-bbf2-0c0fddddb983 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:51.633427 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcc061-ac68-48a0-bbf2-0c0fddddb983 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:51.633736 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcc061-ac68-48a0-bbf2-0c0fddddb983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:51.658514 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3efcc061-ac68-48a0-bbf2-0c0fddddb983 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:51.659725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3efcc061-ac68-48a0-bbf2-0c0fddddb983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:51.667931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1303/2634] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:51.672125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8447423-c65b-4cf8-b15c-6a613df3cf9f None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:51.682395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8447423-c65b-4cf8-b15c-6a613df3cf9f None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:51.687366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab0072-2418-43cc-9b89-f943f3c2f676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:51.688050 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab0072-2418-43cc-9b89-f943f3c2f676 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:51.688386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab0072-2418-43cc-9b89-f943f3c2f676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:51.688710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab0072-2418-43cc-9b89-f943f3c2f676 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:51.689435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab0072-2418-43cc-9b89-f943f3c2f676 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:56:51.689827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab0072-2418-43cc-9b89-f943f3c2f676 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:51.691549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab0072-2418-43cc-9b89-f943f3c2f676 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:51.709329 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1332/2635] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:56:51.710460 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcc061-ac68-48a0-bbf2-0c0fddddb983 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:51.719926 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcc061-ac68-48a0-bbf2-0c0fddddb983 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:51.724804 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a3222-9f80-491f-809e-ba5f5c9f7a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:51.725483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a3222-9f80-491f-809e-ba5f5c9f7a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:51.725970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a3222-9f80-491f-809e-ba5f5c9f7a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:51.726240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a3222-9f80-491f-809e-ba5f5c9f7a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:51.727143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a3222-9f80-491f-809e-ba5f5c9f7a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:51.729330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a3222-9f80-491f-809e-ba5f5c9f7a74 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:51.844994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1304/2636] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:56:51 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:51.848551 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab0072-2418-43cc-9b89-f943f3c2f676 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:51.860493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab0072-2418-43cc-9b89-f943f3c2f676 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:51.918298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=100 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:51.930606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0758e1d-86ff-4e0e-a3aa-c3eb44839007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:51.931254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0758e1d-86ff-4e0e-a3aa-c3eb44839007 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:51.931474 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0758e1d-86ff-4e0e-a3aa-c3eb44839007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:51.931749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0758e1d-86ff-4e0e-a3aa-c3eb44839007 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:51.937694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0758e1d-86ff-4e0e-a3aa-c3eb44839007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:51.937694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0758e1d-86ff-4e0e-a3aa-c3eb44839007 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:52.024178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1333/2637] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:56:51 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:52.025287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a3222-9f80-491f-809e-ba5f5c9f7a74 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:52.042145 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a3222-9f80-491f-809e-ba5f5c9f7a74 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:52.049220 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aec94c-68a5-4d42-ada4-9552deb89ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:52.049880 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aec94c-68a5-4d42-ada4-9552deb89ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:52.050162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aec94c-68a5-4d42-ada4-9552deb89ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:52.050435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aec94c-68a5-4d42-ada4-9552deb89ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:52.051168 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aec94c-68a5-4d42-ada4-9552deb89ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:52.053699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aec94c-68a5-4d42-ada4-9552deb89ef1 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:52.065679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1305/2638] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:56:52.069975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0758e1d-86ff-4e0e-a3aa-c3eb44839007 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:52.082389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0758e1d-86ff-4e0e-a3aa-c3eb44839007 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:52.088698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f944020-23ea-40f9-80b7-275d6bf542ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:52.089346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f944020-23ea-40f9-80b7-275d6bf542ae admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:52.089684 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f944020-23ea-40f9-80b7-275d6bf542ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:52.089956 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f944020-23ea-40f9-80b7-275d6bf542ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:52.090599 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f944020-23ea-40f9-80b7-275d6bf542ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:52.092519 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f944020-23ea-40f9-80b7-275d6bf542ae admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:52.105406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1334/2639] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:52.110262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aec94c-68a5-4d42-ada4-9552deb89ef1 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:52.122204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aec94c-68a5-4d42-ada4-9552deb89ef1 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:52.129239 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d604729-2587-4321-91c4-a7d11db3eb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:52.130799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d604729-2587-4321-91c4-a7d11db3eb13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:52.130799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d604729-2587-4321-91c4-a7d11db3eb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:52.130799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d604729-2587-4321-91c4-a7d11db3eb13 admin admin] PATH_INFO: `/v3/projects/04f783acdcf24801bfe2125366a21956/users/3b98b1573829414b8321708bd02ce53f/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:52.149009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d604729-2587-4321-91c4-a7d11db3eb13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f783acdcf24801bfe2125366a21956, user_id=3b98b1573829414b8321708bd02ce53f, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:52.151809 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d604729-2587-4321-91c4-a7d11db3eb13 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:52.242030 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1306/2640] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:52 2022] PUT /identity/v3/projects/04f783acdcf24801bfe2125366a21956/users/3b98b1573829414b8321708bd02ce53f/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:52.254348 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956404a-c0f6-4572-ae54-70017933490d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:52.254642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956404a-c0f6-4572-ae54-70017933490d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:52.254919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956404a-c0f6-4572-ae54-70017933490d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:52.288750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-f956404a-c0f6-4572-ae54-70017933490d None None] MFA Rules not processed for user `3b98b1573829414b8321708bd02ce53f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:52.290090 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f956404a-c0f6-4572-ae54-70017933490d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:52.383132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1335/2641] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:52.387582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d604729-2587-4321-91c4-a7d11db3eb13 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:52.465590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d604729-2587-4321-91c4-a7d11db3eb13 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:52.470858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529d01b-9f4d-438c-a224-5a7985f094df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:52.471483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529d01b-9f4d-438c-a224-5a7985f094df admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:52.471760 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529d01b-9f4d-438c-a224-5a7985f094df admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:52.472039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529d01b-9f4d-438c-a224-5a7985f094df admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:52.472680 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e529d01b-9f4d-438c-a224-5a7985f094df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:52.475488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e529d01b-9f4d-438c-a224-5a7985f094df admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:52.548791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1307/2642] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:56:52 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:52.550495 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956404a-c0f6-4572-ae54-70017933490d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:52.565934 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956404a-c0f6-4572-ae54-70017933490d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:52.637318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e7db9-942b-41d1-ab33-700aa26b2421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:52.638323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e7db9-942b-41d1-ab33-700aa26b2421 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:52.638602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e7db9-942b-41d1-ab33-700aa26b2421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:52.638933 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e7db9-942b-41d1-ab33-700aa26b2421 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:52.639902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e7db9-942b-41d1-ab33-700aa26b2421 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:52.643537 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e7db9-942b-41d1-ab33-700aa26b2421 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:52.736757 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1336/2643] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:56:52 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:56:52.738152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529d01b-9f4d-438c-a224-5a7985f094df admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:52.749142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529d01b-9f4d-438c-a224-5a7985f094df admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:52.753943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bb475-e37b-4b83-aa24-c02ca7e21590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:52.754621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bb475-e37b-4b83-aa24-c02ca7e21590 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:52.754904 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bb475-e37b-4b83-aa24-c02ca7e21590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:52.755176 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bb475-e37b-4b83-aa24-c02ca7e21590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:52.756748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671bb475-e37b-4b83-aa24-c02ca7e21590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:52.758367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671bb475-e37b-4b83-aa24-c02ca7e21590 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:52.769872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1308/2644] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:52.775853 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e7db9-942b-41d1-ab33-700aa26b2421 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:52.788155 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e7db9-942b-41d1-ab33-700aa26b2421 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:52.794423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af6038-7a1b-4359-a222-5ce65eb51906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:52.794986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af6038-7a1b-4359-a222-5ce65eb51906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:52.795260 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af6038-7a1b-4359-a222-5ce65eb51906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:52.795535 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af6038-7a1b-4359-a222-5ce65eb51906 admin admin] PATH_INFO: `/v3/projects/0f31981050424188806b5a52a2bae903/users/cc154c94ae8d4cf8a0c1a96ba0fbbd67/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:52.815057 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af6038-7a1b-4359-a222-5ce65eb51906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f31981050424188806b5a52a2bae903, user_id=cc154c94ae8d4cf8a0c1a96ba0fbbd67, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:52.819133 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af6038-7a1b-4359-a222-5ce65eb51906 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:52.882656 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1337/2645] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:52 2022] PUT /identity/v3/projects/0f31981050424188806b5a52a2bae903/users/cc154c94ae8d4cf8a0c1a96ba0fbbd67/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:56:52.886722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bb475-e37b-4b83-aa24-c02ca7e21590 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:52.897448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bb475-e37b-4b83-aa24-c02ca7e21590 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:52.926576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:52.961028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7511fc-5ea6-4b56-a73a-557bc9a864e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:52.961677 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7511fc-5ea6-4b56-a73a-557bc9a864e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:52.961961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7511fc-5ea6-4b56-a73a-557bc9a864e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:52.962356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7511fc-5ea6-4b56-a73a-557bc9a864e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:52.962946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7511fc-5ea6-4b56-a73a-557bc9a864e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:52.964576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7511fc-5ea6-4b56-a73a-557bc9a864e2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:52.975626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1309/2646] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:56:52.979558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af6038-7a1b-4359-a222-5ce65eb51906 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:52.991321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af6038-7a1b-4359-a222-5ce65eb51906 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:52.996365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff5cab-c346-4f76-af74-3a5d144925a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:52.997267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff5cab-c346-4f76-af74-3a5d144925a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:52.997654 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff5cab-c346-4f76-af74-3a5d144925a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:52.998610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff5cab-c346-4f76-af74-3a5d144925a7 admin admin] PATH_INFO: `/v3/projects/0f31981050424188806b5a52a2bae903/users/cc154c94ae8d4cf8a0c1a96ba0fbbd67/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:53.001401 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff5cab-c346-4f76-af74-3a5d144925a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f31981050424188806b5a52a2bae903, user_id=cc154c94ae8d4cf8a0c1a96ba0fbbd67, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:53.004425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff5cab-c346-4f76-af74-3a5d144925a7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:53.016024 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1338/2647] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:56:52 2022] PUT /identity/v3/projects/0f31981050424188806b5a52a2bae903/users/cc154c94ae8d4cf8a0c1a96ba0fbbd67/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:56:53.026380 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ff27a-9727-42a4-a07b-23d490ef562c None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:53.026663 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ff27a-9727-42a4-a07b-23d490ef562c None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:53.026938 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ff27a-9727-42a4-a07b-23d490ef562c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:53.059110 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9a7ff27a-9727-42a4-a07b-23d490ef562c None None] MFA Rules not processed for user `cc154c94ae8d4cf8a0c1a96ba0fbbd67`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:56:53.060495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9a7ff27a-9727-42a4-a07b-23d490ef562c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:53.131953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1310/2648] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:56:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:56:53.150208 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff5cab-c346-4f76-af74-3a5d144925a7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:53.152677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2dff5cab-c346-4f76-af74-3a5d144925a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:56:53.173480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:56:53.238803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff5cab-c346-4f76-af74-3a5d144925a7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:53.243869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013aefb-fd14-442e-98cf-eb5106cbac09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:53.244509 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013aefb-fd14-442e-98cf-eb5106cbac09 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:53.244913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013aefb-fd14-442e-98cf-eb5106cbac09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:53.245349 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013aefb-fd14-442e-98cf-eb5106cbac09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:53.258363 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013aefb-fd14-442e-98cf-eb5106cbac09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:53.259809 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013aefb-fd14-442e-98cf-eb5106cbac09 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:53.271036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1339/2649] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:56:53.596833 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ff27a-9727-42a4-a07b-23d490ef562c None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:56:53.608244 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ff27a-9727-42a4-a07b-23d490ef562c None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:56:53.611983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f526daf-f3b8-4ed6-94d1-686558326003 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:56:53.612644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f526daf-f3b8-4ed6-94d1-686558326003 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:56:53.612912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f526daf-f3b8-4ed6-94d1-686558326003 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:56:53.613184 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f526daf-f3b8-4ed6-94d1-686558326003 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:56:53.674676 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f526daf-f3b8-4ed6-94d1-686558326003 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:56:53.676308 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f526daf-f3b8-4ed6-94d1-686558326003 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:56:53.686481 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1311/2650] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:56:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:01.201372 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013aefb-fd14-442e-98cf-eb5106cbac09 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:01.273950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013aefb-fd14-442e-98cf-eb5106cbac09 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:01.278456 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:01.279087 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:01.279366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:01.279636 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] PATH_INFO: `/v3/users/3b98b1573829414b8321708bd02ce53f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:01.280775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b98b1573829414b8321708bd02ce53f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:01.282717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:01.468883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b98b1573829414b8321708bd02ce53f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:01.538224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b98b1573829414b8321708bd02ce53f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:01.539126 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b98b1573829414b8321708bd02ce53f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:01.547464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b98b1573829414b8321708bd02ce53f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:01.560320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b98b1573829414b8321708bd02ce53f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:01.567159 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1340/2651] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:01 2022] DELETE /identity/v3/users/3b98b1573829414b8321708bd02ce53f => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:01.572317 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faf448-33bd-49f3-88ba-dffafe56d114 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:01.572824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faf448-33bd-49f3-88ba-dffafe56d114 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:01.573777 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faf448-33bd-49f3-88ba-dffafe56d114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:01.594906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-53faf448-33bd-49f3-88ba-dffafe56d114 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:01.596128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-53faf448-33bd-49f3-88ba-dffafe56d114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:01.646448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:01.666105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1312/2652] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:01.677493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:01.678927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:01.716991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:01.738494 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a24e5-a822-4169-b368-9f860482deb2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:01.742173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667480-0ccb-4685-8932-cc6640110161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:01.742770 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667480-0ccb-4685-8932-cc6640110161 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:01.743086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667480-0ccb-4685-8932-cc6640110161 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:01.743419 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667480-0ccb-4685-8932-cc6640110161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:01.752452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667480-0ccb-4685-8932-cc6640110161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:01.753918 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667480-0ccb-4685-8932-cc6640110161 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:01.764459 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1341/2653] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:02.225254 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faf448-33bd-49f3-88ba-dffafe56d114 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:02.227375 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-53faf448-33bd-49f3-88ba-dffafe56d114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:02.242432 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faf448-33bd-49f3-88ba-dffafe56d114 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:02.247207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:02.247855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:02.248140 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:02.248413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] PATH_INFO: `/v3/projects/04f783acdcf24801bfe2125366a21956` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:02.249583 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f783acdcf24801bfe2125366a21956)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:02.251506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:02.389821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f783acdcf24801bfe2125366a21956', 'request_id': 'req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:02.396532 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f783acdcf24801bfe2125366a21956', 'request_id': 'req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:02.402204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f783acdcf24801bfe2125366a21956', 'request_id': 'req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:02.463248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] The token cache is being invalidate because project 04f783acdcf24801bfe2125366a21956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:02.463549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:02.464162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:02.465915 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1313/2654] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:02 2022] DELETE /identity/v3/projects/04f783acdcf24801bfe2125366a21956 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:02.469675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667480-0ccb-4685-8932-cc6640110161 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:02.482662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667480-0ccb-4685-8932-cc6640110161 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:02.535847 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87d8a2-95b3-482d-9940-98576c30ac7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:02.536467 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87d8a2-95b3-482d-9940-98576c30ac7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:02.536745 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87d8a2-95b3-482d-9940-98576c30ac7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:02.537014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87d8a2-95b3-482d-9940-98576c30ac7b admin admin] PATH_INFO: `/v3/users/cc154c94ae8d4cf8a0c1a96ba0fbbd67` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:02.538817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87d8a2-95b3-482d-9940-98576c30ac7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc154c94ae8d4cf8a0c1a96ba0fbbd67)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:02.540967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87d8a2-95b3-482d-9940-98576c30ac7b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:02.688799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8c87d8a2-95b3-482d-9940-98576c30ac7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc154c94ae8d4cf8a0c1a96ba0fbbd67'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:02.744098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8c87d8a2-95b3-482d-9940-98576c30ac7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc154c94ae8d4cf8a0c1a96ba0fbbd67'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:02.744720 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8c87d8a2-95b3-482d-9940-98576c30ac7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc154c94ae8d4cf8a0c1a96ba0fbbd67'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:02.752110 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8c87d8a2-95b3-482d-9940-98576c30ac7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc154c94ae8d4cf8a0c1a96ba0fbbd67'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:02.764515 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8c87d8a2-95b3-482d-9940-98576c30ac7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc154c94ae8d4cf8a0c1a96ba0fbbd67'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:02.770454 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1342/2655] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:02 2022] DELETE /identity/v3/users/cc154c94ae8d4cf8a0c1a96ba0fbbd67 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:03.219389 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:03.220849 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:03.233980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8950a58-8408-4bee-8ad7-cc20d80f8d2c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:03.305298 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382488f9-749c-432c-bf6a-585863fbf259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:03.305975 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382488f9-749c-432c-bf6a-585863fbf259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:03.306258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382488f9-749c-432c-bf6a-585863fbf259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:03.306535 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382488f9-749c-432c-bf6a-585863fbf259 admin admin] PATH_INFO: `/v3/projects/0f31981050424188806b5a52a2bae903` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:03.307746 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382488f9-749c-432c-bf6a-585863fbf259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f31981050424188806b5a52a2bae903)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:03.309678 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382488f9-749c-432c-bf6a-585863fbf259 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:03.432362 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-382488f9-749c-432c-bf6a-585863fbf259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f31981050424188806b5a52a2bae903', 'request_id': 'req-382488f9-749c-432c-bf6a-585863fbf259'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:03.439616 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-382488f9-749c-432c-bf6a-585863fbf259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f31981050424188806b5a52a2bae903', 'request_id': 'req-382488f9-749c-432c-bf6a-585863fbf259'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:03.446644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-382488f9-749c-432c-bf6a-585863fbf259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f31981050424188806b5a52a2bae903', 'request_id': 'req-382488f9-749c-432c-bf6a-585863fbf259'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:03.555635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-382488f9-749c-432c-bf6a-585863fbf259 admin admin] The token cache is being invalidate because project 0f31981050424188806b5a52a2bae903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:03.559718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-382488f9-749c-432c-bf6a-585863fbf259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:03.559718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-382488f9-749c-432c-bf6a-585863fbf259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:03.559718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1314/2656] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:03 2022] DELETE /identity/v3/projects/0f31981050424188806b5a52a2bae903 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:03.563192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f87cd-4f95-42b0-a25a-9a364092bb46 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:03.563645 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f87cd-4f95-42b0-a25a-9a364092bb46 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:03.563750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f87cd-4f95-42b0-a25a-9a364092bb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:03.583739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c36f87cd-4f95-42b0-a25a-9a364092bb46 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:03.584938 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c36f87cd-4f95-42b0-a25a-9a364092bb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:03.633299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:03.644941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1343/2657] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:03.654092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d1bd-48a5-4888-b4ee-f87a60b2894b None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:03.654382 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d1bd-48a5-4888-b4ee-f87a60b2894b None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:03.654655 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d1bd-48a5-4888-b4ee-f87a60b2894b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:03.662421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:03.674589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9b43d1bd-48a5-4888-b4ee-f87a60b2894b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:03.675819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9b43d1bd-48a5-4888-b4ee-f87a60b2894b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:03.684231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1315/2658] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:03.688840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f87cd-4f95-42b0-a25a-9a364092bb46 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:03.699353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f87cd-4f95-42b0-a25a-9a364092bb46 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:03.704203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d0366-dab0-4447-a1c2-089d071358ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:03.704900 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d0366-dab0-4447-a1c2-089d071358ed admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:03.705183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d0366-dab0-4447-a1c2-089d071358ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:03.705454 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d0366-dab0-4447-a1c2-089d071358ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:03.706138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d0366-dab0-4447-a1c2-089d071358ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:57:03.706442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d0366-dab0-4447-a1c2-089d071358ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:03.708071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d0366-dab0-4447-a1c2-089d071358ed admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:03.724959 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1344/2659] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:57:03.726882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d1bd-48a5-4888-b4ee-f87a60b2894b None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:03.736491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d1bd-48a5-4888-b4ee-f87a60b2894b None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:03.744029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d4296-2036-4447-b477-f4ed04c0e241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:03.744029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d4296-2036-4447-b477-f4ed04c0e241 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:03.744029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d4296-2036-4447-b477-f4ed04c0e241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:03.744029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d4296-2036-4447-b477-f4ed04c0e241 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:03.744492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d4296-2036-4447-b477-f4ed04c0e241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:03.746210 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d4296-2036-4447-b477-f4ed04c0e241 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:03.869315 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1316/2660] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:57:03 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:57:03.870470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d0366-dab0-4447-a1c2-089d071358ed admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:03.880521 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d0366-dab0-4447-a1c2-089d071358ed admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:03.939147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22805e-dda5-4360-8575-5cd4e667bf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:03.939781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22805e-dda5-4360-8575-5cd4e667bf96 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:03.940062 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22805e-dda5-4360-8575-5cd4e667bf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:03.940337 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22805e-dda5-4360-8575-5cd4e667bf96 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:03.940969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22805e-dda5-4360-8575-5cd4e667bf96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:03.942950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22805e-dda5-4360-8575-5cd4e667bf96 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:04.025043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1345/2661] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:57:03 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:57:04.028357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d4296-2036-4447-b477-f4ed04c0e241 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:04.039900 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d4296-2036-4447-b477-f4ed04c0e241 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:04.044501 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410b885-124c-41c2-8825-221b210e8d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:04.047758 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410b885-124c-41c2-8825-221b210e8d48 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:04.047758 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410b885-124c-41c2-8825-221b210e8d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:04.047758 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410b885-124c-41c2-8825-221b210e8d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:04.047758 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410b885-124c-41c2-8825-221b210e8d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:04.048296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410b885-124c-41c2-8825-221b210e8d48 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:04.058437 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1317/2662] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:04.062918 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22805e-dda5-4360-8575-5cd4e667bf96 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:04.073357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22805e-dda5-4360-8575-5cd4e667bf96 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:04.078617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdb907-f289-442a-881a-63c7de24a1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:04.079274 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdb907-f289-442a-881a-63c7de24a1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:04.079557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdb907-f289-442a-881a-63c7de24a1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:04.079832 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdb907-f289-442a-881a-63c7de24a1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:04.080475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebdb907-f289-442a-881a-63c7de24a1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:04.082230 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebdb907-f289-442a-881a-63c7de24a1d0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:04.093143 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1346/2663] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:04.097511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410b885-124c-41c2-8825-221b210e8d48 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:04.108059 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410b885-124c-41c2-8825-221b210e8d48 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:04.112734 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eadaa0-59cc-4ecb-bec7-14756b5ef04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:04.113377 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eadaa0-59cc-4ecb-bec7-14756b5ef04a admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:04.113725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eadaa0-59cc-4ecb-bec7-14756b5ef04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:04.116818 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eadaa0-59cc-4ecb-bec7-14756b5ef04a admin admin] PATH_INFO: `/v3/projects/9b748fb301734bf4a5562058d3f35571/users/40f0aea1961e423bbffcd99cd3fe0f22/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:04.129343 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eadaa0-59cc-4ecb-bec7-14756b5ef04a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b748fb301734bf4a5562058d3f35571, user_id=40f0aea1961e423bbffcd99cd3fe0f22, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:04.132060 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eadaa0-59cc-4ecb-bec7-14756b5ef04a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:04.224166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1318/2664] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:04 2022] PUT /identity/v3/projects/9b748fb301734bf4a5562058d3f35571/users/40f0aea1961e423bbffcd99cd3fe0f22/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:04.240812 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfff2a-0203-41ba-993c-0886125b1afb None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:04.241129 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfff2a-0203-41ba-993c-0886125b1afb None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:04.241403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfff2a-0203-41ba-993c-0886125b1afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:04.270620 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-36cfff2a-0203-41ba-993c-0886125b1afb None None] MFA Rules not processed for user `40f0aea1961e423bbffcd99cd3fe0f22`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:04.272030 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-36cfff2a-0203-41ba-993c-0886125b1afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:04.341192 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1347/2665] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:04.346210 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eadaa0-59cc-4ecb-bec7-14756b5ef04a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:04.410144 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eadaa0-59cc-4ecb-bec7-14756b5ef04a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:04.415442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a480bcd-96b3-4d24-9d4f-c498edcb3ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:04.416035 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a480bcd-96b3-4d24-9d4f-c498edcb3ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:04.416312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a480bcd-96b3-4d24-9d4f-c498edcb3ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:04.416590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a480bcd-96b3-4d24-9d4f-c498edcb3ec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:04.417208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a480bcd-96b3-4d24-9d4f-c498edcb3ec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:04.419127 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a480bcd-96b3-4d24-9d4f-c498edcb3ec6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:04.540037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1319/2666] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:57:04 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:04.543958 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfff2a-0203-41ba-993c-0886125b1afb None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:04.554779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfff2a-0203-41ba-993c-0886125b1afb None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:04.602665 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:04.613125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c323f0-6d8c-4357-8624-f318f133351c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:04.613780 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c323f0-6d8c-4357-8624-f318f133351c admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:04.615601 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c323f0-6d8c-4357-8624-f318f133351c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:04.615601 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c323f0-6d8c-4357-8624-f318f133351c admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:04.615601 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c323f0-6d8c-4357-8624-f318f133351c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:04.617056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c323f0-6d8c-4357-8624-f318f133351c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:04.683647 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1348/2667] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:57:04 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:04.685418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a480bcd-96b3-4d24-9d4f-c498edcb3ec6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:04.696986 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a480bcd-96b3-4d24-9d4f-c498edcb3ec6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:04.701754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef22c7-067e-4664-ab7b-742f1ee838ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:04.703193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef22c7-067e-4664-ab7b-742f1ee838ab admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:04.703193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef22c7-067e-4664-ab7b-742f1ee838ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:04.703193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef22c7-067e-4664-ab7b-742f1ee838ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:04.703630 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef22c7-067e-4664-ab7b-742f1ee838ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:04.705091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef22c7-067e-4664-ab7b-742f1ee838ab admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:04.716795 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1320/2668] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:04.720914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c323f0-6d8c-4357-8624-f318f133351c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:04.731355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c323f0-6d8c-4357-8624-f318f133351c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:04.736128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b85b2b-cf59-4e55-8bc0-391230c8d954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:04.739803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b85b2b-cf59-4e55-8bc0-391230c8d954 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:04.739803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b85b2b-cf59-4e55-8bc0-391230c8d954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:04.739803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b85b2b-cf59-4e55-8bc0-391230c8d954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:04.739803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b85b2b-cf59-4e55-8bc0-391230c8d954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:04.739803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b85b2b-cf59-4e55-8bc0-391230c8d954 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:04.750734 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1349/2669] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:04.755206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef22c7-067e-4664-ab7b-742f1ee838ab admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:04.766437 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef22c7-067e-4664-ab7b-742f1ee838ab admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:04.771189 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e62c1-4193-40cc-b0cf-02775d78234b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:04.774400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e62c1-4193-40cc-b0cf-02775d78234b admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:04.774400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e62c1-4193-40cc-b0cf-02775d78234b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:04.774400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e62c1-4193-40cc-b0cf-02775d78234b admin admin] PATH_INFO: `/v3/projects/2e341174b33b4bb8b171ec9883d13e8b/users/26422b42dfd04512855cc6defcec3a6c/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:04.788723 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e62c1-4193-40cc-b0cf-02775d78234b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e341174b33b4bb8b171ec9883d13e8b, user_id=26422b42dfd04512855cc6defcec3a6c, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:04.791491 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e62c1-4193-40cc-b0cf-02775d78234b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:04.887790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1321/2670] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:04 2022] PUT /identity/v3/projects/2e341174b33b4bb8b171ec9883d13e8b/users/26422b42dfd04512855cc6defcec3a6c/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:04.898059 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33096e35-9568-41ef-87b0-82a26bb25a71 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:04.898388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33096e35-9568-41ef-87b0-82a26bb25a71 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:04.898617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33096e35-9568-41ef-87b0-82a26bb25a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:04.928005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-33096e35-9568-41ef-87b0-82a26bb25a71 None None] MFA Rules not processed for user `26422b42dfd04512855cc6defcec3a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:04.929790 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-33096e35-9568-41ef-87b0-82a26bb25a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:04.996031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1350/2671] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:05.000545 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e62c1-4193-40cc-b0cf-02775d78234b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:05.058985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:05.077670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e62c1-4193-40cc-b0cf-02775d78234b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:05.078207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31db5f-db3e-47bc-9e77-92bd07e4c588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:05.078814 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31db5f-db3e-47bc-9e77-92bd07e4c588 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:05.079101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31db5f-db3e-47bc-9e77-92bd07e4c588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:05.079373 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31db5f-db3e-47bc-9e77-92bd07e4c588 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:05.079997 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31db5f-db3e-47bc-9e77-92bd07e4c588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:05.081927 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31db5f-db3e-47bc-9e77-92bd07e4c588 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:05.196308 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1322/2672] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:57:04 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:05.200460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33096e35-9568-41ef-87b0-82a26bb25a71 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:05.209714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33096e35-9568-41ef-87b0-82a26bb25a71 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:05.278971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d18e1-f0cb-4323-826a-84b4e53b3198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:05.279650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d18e1-f0cb-4323-826a-84b4e53b3198 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:05.279882 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d18e1-f0cb-4323-826a-84b4e53b3198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:05.280157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d18e1-f0cb-4323-826a-84b4e53b3198 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:05.280803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d18e1-f0cb-4323-826a-84b4e53b3198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:05.282806 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d18e1-f0cb-4323-826a-84b4e53b3198 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:05.369436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1351/2673] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:57:05 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:05.373799 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31db5f-db3e-47bc-9e77-92bd07e4c588 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:05.388869 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31db5f-db3e-47bc-9e77-92bd07e4c588 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:05.394364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520d6d1-676f-4928-9e17-cb2941214dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:05.395070 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520d6d1-676f-4928-9e17-cb2941214dee admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:05.395564 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520d6d1-676f-4928-9e17-cb2941214dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:05.395906 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520d6d1-676f-4928-9e17-cb2941214dee admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:05.396767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520d6d1-676f-4928-9e17-cb2941214dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:05.399209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520d6d1-676f-4928-9e17-cb2941214dee admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:05.410085 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1323/2674] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:05.414799 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d18e1-f0cb-4323-826a-84b4e53b3198 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:05.426005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d18e1-f0cb-4323-826a-84b4e53b3198 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:05.431498 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768c3f2-197a-4162-940b-cc293704d259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:05.436893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768c3f2-197a-4162-940b-cc293704d259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:05.436893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768c3f2-197a-4162-940b-cc293704d259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:05.436893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768c3f2-197a-4162-940b-cc293704d259 admin admin] PATH_INFO: `/v3/projects/dd6bca51fdaf415a82632d1f342f2da0/users/73bb97aa631a4a09ae9086da1d1c085d/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:05.448857 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768c3f2-197a-4162-940b-cc293704d259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6bca51fdaf415a82632d1f342f2da0, user_id=73bb97aa631a4a09ae9086da1d1c085d, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:05.451640 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768c3f2-197a-4162-940b-cc293704d259 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:05.532500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1352/2675] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:05 2022] PUT /identity/v3/projects/dd6bca51fdaf415a82632d1f342f2da0/users/73bb97aa631a4a09ae9086da1d1c085d/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:05.536699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520d6d1-676f-4928-9e17-cb2941214dee admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:05.547340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520d6d1-676f-4928-9e17-cb2941214dee admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:05.606243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee642b1-6ba9-4ae8-83f6-77a6ab489c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:05.606855 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee642b1-6ba9-4ae8-83f6-77a6ab489c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:05.607130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee642b1-6ba9-4ae8-83f6-77a6ab489c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:05.607400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee642b1-6ba9-4ae8-83f6-77a6ab489c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:05.608036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee642b1-6ba9-4ae8-83f6-77a6ab489c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:05.609636 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee642b1-6ba9-4ae8-83f6-77a6ab489c4e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:05.620152 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1324/2676] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:05.624523 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768c3f2-197a-4162-940b-cc293704d259 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:05.635446 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768c3f2-197a-4162-940b-cc293704d259 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:05.640240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a618d8f-b926-454e-adfa-ee6e9a6cc9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:05.640878 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a618d8f-b926-454e-adfa-ee6e9a6cc9dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:05.641157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a618d8f-b926-454e-adfa-ee6e9a6cc9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:05.641427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a618d8f-b926-454e-adfa-ee6e9a6cc9dc admin admin] PATH_INFO: `/v3/projects/dd6bca51fdaf415a82632d1f342f2da0/users/73bb97aa631a4a09ae9086da1d1c085d/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:05.643581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a618d8f-b926-454e-adfa-ee6e9a6cc9dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6bca51fdaf415a82632d1f342f2da0, user_id=73bb97aa631a4a09ae9086da1d1c085d, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:05.647975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a618d8f-b926-454e-adfa-ee6e9a6cc9dc admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:05.657358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1353/2677] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:05 2022] PUT /identity/v3/projects/dd6bca51fdaf415a82632d1f342f2da0/users/73bb97aa631a4a09ae9086da1d1c085d/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:05.738703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35adb75-d3d2-432e-a028-f128ed34f761 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:05.738993 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35adb75-d3d2-432e-a028-f128ed34f761 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:05.739279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35adb75-d3d2-432e-a028-f128ed34f761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:05.771028 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d35adb75-d3d2-432e-a028-f128ed34f761 None None] MFA Rules not processed for user `73bb97aa631a4a09ae9086da1d1c085d`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:05.772780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d35adb75-d3d2-432e-a028-f128ed34f761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:05.842025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1325/2678] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:05.854000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a618d8f-b926-454e-adfa-ee6e9a6cc9dc admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:05.855703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6a618d8f-b926-454e-adfa-ee6e9a6cc9dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:05.869651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:05.914424 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a618d8f-b926-454e-adfa-ee6e9a6cc9dc admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:05.918025 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d15196-ee92-4dc5-916f-6984e8ea500c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:05.918625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d15196-ee92-4dc5-916f-6984e8ea500c service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:05.918903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d15196-ee92-4dc5-916f-6984e8ea500c service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:05.919178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d15196-ee92-4dc5-916f-6984e8ea500c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:05.974758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d15196-ee92-4dc5-916f-6984e8ea500c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:05.976827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d15196-ee92-4dc5-916f-6984e8ea500c service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:05.986340 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1354/2679] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:07.952969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35adb75-d3d2-432e-a028-f128ed34f761 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:07.952969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d35adb75-d3d2-432e-a028-f128ed34f761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:07.963213 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35adb75-d3d2-432e-a028-f128ed34f761 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:07.966718 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d105716-54e9-4d61-80f8-cdd6e03c1b77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:07.967312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d105716-54e9-4d61-80f8-cdd6e03c1b77 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:07.967590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d105716-54e9-4d61-80f8-cdd6e03c1b77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:07.967859 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d105716-54e9-4d61-80f8-cdd6e03c1b77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:08.009282 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:08.023977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d105716-54e9-4d61-80f8-cdd6e03c1b77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:08.025417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d105716-54e9-4d61-80f8-cdd6e03c1b77 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:08.034374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1326/2680] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:10.500894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d15196-ee92-4dc5-916f-6984e8ea500c service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:10.519708 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d15196-ee92-4dc5-916f-6984e8ea500c service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:10.523977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fad6fa-3252-464c-831a-6a6ee3a9ee84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:10.524691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fad6fa-3252-464c-831a-6a6ee3a9ee84 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:10.524981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fad6fa-3252-464c-831a-6a6ee3a9ee84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:10.525250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fad6fa-3252-464c-831a-6a6ee3a9ee84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:10.534525 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fad6fa-3252-464c-831a-6a6ee3a9ee84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:10.535943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fad6fa-3252-464c-831a-6a6ee3a9ee84 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:10.545331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1355/2681] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:17.185637 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d105716-54e9-4d61-80f8-cdd6e03c1b77 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:17.252904 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d105716-54e9-4d61-80f8-cdd6e03c1b77 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:17.257399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:17.258582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:17.258862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:17.259134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] PATH_INFO: `/v3/users/40f0aea1961e423bbffcd99cd3fe0f22` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:17.264576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40f0aea1961e423bbffcd99cd3fe0f22)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:17.264576 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:17.458774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40f0aea1961e423bbffcd99cd3fe0f22'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:17.555814 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40f0aea1961e423bbffcd99cd3fe0f22'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:17.556509 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40f0aea1961e423bbffcd99cd3fe0f22'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:17.567241 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40f0aea1961e423bbffcd99cd3fe0f22'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:17.580157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40f0aea1961e423bbffcd99cd3fe0f22'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:17.587075 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1327/2682] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:17 2022] DELETE /identity/v3/users/40f0aea1961e423bbffcd99cd3fe0f22 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:17.597305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bfdbf-cc7d-4cd5-a63d-d308803bd8af None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:17.597820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bfdbf-cc7d-4cd5-a63d-d308803bd8af None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:17.597943 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bfdbf-cc7d-4cd5-a63d-d308803bd8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:17.618386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8b6bfdbf-cc7d-4cd5-a63d-d308803bd8af None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:17.619710 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8b6bfdbf-cc7d-4cd5-a63d-d308803bd8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:17.685886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1356/2683] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:57:17.697011 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:17.698443 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:17.773357 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:17.783365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b209c6-59ca-4cca-ad37-91477f44ea59 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:17.787390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b1df3-7be9-49db-b578-6d352ef9604e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:17.788003 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b1df3-7be9-49db-b578-6d352ef9604e service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:17.788279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b1df3-7be9-49db-b578-6d352ef9604e service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:17.788546 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b1df3-7be9-49db-b578-6d352ef9604e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:17.799389 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b1df3-7be9-49db-b578-6d352ef9604e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:17.801444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b1df3-7be9-49db-b578-6d352ef9604e service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:17.812654 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1328/2684] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:18.129028 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bfdbf-cc7d-4cd5-a63d-d308803bd8af None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:18.130534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8b6bfdbf-cc7d-4cd5-a63d-d308803bd8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:18.142820 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bfdbf-cc7d-4cd5-a63d-d308803bd8af None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:18.147433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68e62d-988c-4195-8139-747fcc4f5e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:18.148128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68e62d-988c-4195-8139-747fcc4f5e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:18.148436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68e62d-988c-4195-8139-747fcc4f5e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:18.148707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68e62d-988c-4195-8139-747fcc4f5e85 admin admin] PATH_INFO: `/v3/users/6ed400cb07b84176b7bc0c5b56284695` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:18.149858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68e62d-988c-4195-8139-747fcc4f5e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ed400cb07b84176b7bc0c5b56284695)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:18.154422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68e62d-988c-4195-8139-747fcc4f5e85 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:18.219665 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b1df3-7be9-49db-b578-6d352ef9604e service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:18.221090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-673b1df3-7be9-49db-b578-6d352ef9604e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:18.241329 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b1df3-7be9-49db-b578-6d352ef9604e service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:18.242857 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:18.243806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:18.244188 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:18.244582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] PATH_INFO: `/v3/projects/9b748fb301734bf4a5562058d3f35571` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:18.245923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b748fb301734bf4a5562058d3f35571)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:18.247871 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:18.342545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:18.348577 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8d68e62d-988c-4195-8139-747fcc4f5e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ed400cb07b84176b7bc0c5b56284695'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.379569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b748fb301734bf4a5562058d3f35571', 'request_id': 'req-1794a931-1a42-4c9f-bbf8-82f81e1f2734'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.385377 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b748fb301734bf4a5562058d3f35571', 'request_id': 'req-1794a931-1a42-4c9f-bbf8-82f81e1f2734'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.391073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b748fb301734bf4a5562058d3f35571', 'request_id': 'req-1794a931-1a42-4c9f-bbf8-82f81e1f2734'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.435026 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8d68e62d-988c-4195-8139-747fcc4f5e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ed400cb07b84176b7bc0c5b56284695'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.435675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8d68e62d-988c-4195-8139-747fcc4f5e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ed400cb07b84176b7bc0c5b56284695'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.443328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8d68e62d-988c-4195-8139-747fcc4f5e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ed400cb07b84176b7bc0c5b56284695'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.455203 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-8d68e62d-988c-4195-8139-747fcc4f5e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ed400cb07b84176b7bc0c5b56284695'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.461743 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1357/2685] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:18 2022] DELETE /identity/v3/users/6ed400cb07b84176b7bc0c5b56284695 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:18.466117 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0a945-0663-4540-b43f-ba6ce28c6a1b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:18.466350 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0a945-0663-4540-b43f-ba6ce28c6a1b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:18.466628 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0a945-0663-4540-b43f-ba6ce28c6a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:18.485844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5bc0a945-0663-4540-b43f-ba6ce28c6a1b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:18.487151 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5bc0a945-0663-4540-b43f-ba6ce28c6a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:18.498970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] The token cache is being invalidate because project 9b748fb301734bf4a5562058d3f35571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:18.499276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.499918 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.501471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1329/2686] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:18 2022] DELETE /identity/v3/projects/9b748fb301734bf4a5562058d3f35571 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:57:18.505899 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:18.508928 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:18.551175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1358/2687] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:57:18.560929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0a945-0663-4540-b43f-ba6ce28c6a1b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:18.562662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5bc0a945-0663-4540-b43f-ba6ce28c6a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:18.589824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794a931-1a42-4c9f-bbf8-82f81e1f2734 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:18.594644 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:18.595270 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:18.595555 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:18.595829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] PATH_INFO: `/v3/users/26422b42dfd04512855cc6defcec3a6c` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:18.596958 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=26422b42dfd04512855cc6defcec3a6c)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:18.598903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:18.620877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0a945-0663-4540-b43f-ba6ce28c6a1b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:18.624879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01018c0c-eaab-4765-a6e3-54642d98a188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:18.625490 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01018c0c-eaab-4765-a6e3-54642d98a188 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:18.625805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01018c0c-eaab-4765-a6e3-54642d98a188 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:18.626078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01018c0c-eaab-4765-a6e3-54642d98a188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:18.635402 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01018c0c-eaab-4765-a6e3-54642d98a188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:18.636845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01018c0c-eaab-4765-a6e3-54642d98a188 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:18.646575 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1359/2688] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:57:18.751939 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26422b42dfd04512855cc6defcec3a6c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.812324 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26422b42dfd04512855cc6defcec3a6c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.812911 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26422b42dfd04512855cc6defcec3a6c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.821022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26422b42dfd04512855cc6defcec3a6c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.826813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:18.833920 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26422b42dfd04512855cc6defcec3a6c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:18.840793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1330/2689] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:18 2022] DELETE /identity/v3/users/26422b42dfd04512855cc6defcec3a6c => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:57:19.037078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01018c0c-eaab-4765-a6e3-54642d98a188 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:19.038877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-01018c0c-eaab-4765-a6e3-54642d98a188 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:19.056272 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01018c0c-eaab-4765-a6e3-54642d98a188 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:19.121623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:19.122357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:19.122723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:19.123046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] PATH_INFO: `/v3/projects/d28aa4e2ac774574b4f72ea8cb08334b` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:19.124255 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d28aa4e2ac774574b4f72ea8cb08334b)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:19.126378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:19.127958 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:19.242756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:19.244449 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:19.251911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd28aa4e2ac774574b4f72ea8cb08334b', 'request_id': 'req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.258314 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c46e71-d677-4c42-9c59-e4e3974c32ca admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:19.268917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd28aa4e2ac774574b4f72ea8cb08334b', 'request_id': 'req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.276631 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd28aa4e2ac774574b4f72ea8cb08334b', 'request_id': 'req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.322201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:19.322819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:19.323095 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:19.323366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] PATH_INFO: `/v3/projects/2e341174b33b4bb8b171ec9883d13e8b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:19.324465 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e341174b33b4bb8b171ec9883d13e8b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:19.326438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:19.349244 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] The token cache is being invalidate because project d28aa4e2ac774574b4f72ea8cb08334b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:19.349554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.350242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.352569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1360/2690] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:19 2022] DELETE /identity/v3/projects/d28aa4e2ac774574b4f72ea8cb08334b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:19.356638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:19.357984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:19.370095 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dcdf0-b9c2-4c10-80c7-1a4adbf5d2b2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:19.374557 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:19.375161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:19.375435 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:19.375701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] PATH_INFO: `/v3/users/2e713a1fd64244b9a772dbc208c995a2` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:19.376848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e713a1fd64244b9a772dbc208c995a2)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:19.378784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:19.450893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e341174b33b4bb8b171ec9883d13e8b', 'request_id': 'req-d6440884-f4a7-4e5e-978d-b4501f52728e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.457206 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e341174b33b4bb8b171ec9883d13e8b', 'request_id': 'req-d6440884-f4a7-4e5e-978d-b4501f52728e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.463759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e341174b33b4bb8b171ec9883d13e8b', 'request_id': 'req-d6440884-f4a7-4e5e-978d-b4501f52728e'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.563407 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e713a1fd64244b9a772dbc208c995a2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.568951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] The token cache is being invalidate because project 2e341174b33b4bb8b171ec9883d13e8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:19.569249 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.569917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.571924 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1331/2691] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:19 2022] DELETE /identity/v3/projects/2e341174b33b4bb8b171ec9883d13e8b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:57:19.575905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:19.577258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:19.636065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e713a1fd64244b9a772dbc208c995a2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.636883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e713a1fd64244b9a772dbc208c995a2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.645747 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e713a1fd64244b9a772dbc208c995a2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.646629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6440884-f4a7-4e5e-978d-b4501f52728e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:19.652219 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df713e-efaf-4eee-b660-08145fa872ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:19.655078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df713e-efaf-4eee-b660-08145fa872ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:19.655078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df713e-efaf-4eee-b660-08145fa872ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:19.655078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df713e-efaf-4eee-b660-08145fa872ec admin admin] PATH_INFO: `/v3/users/73bb97aa631a4a09ae9086da1d1c085d` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:19.655078 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df713e-efaf-4eee-b660-08145fa872ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=73bb97aa631a4a09ae9086da1d1c085d)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:19.662608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df713e-efaf-4eee-b660-08145fa872ec admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:19.662728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e713a1fd64244b9a772dbc208c995a2'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.666116 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1361/2692] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:19 2022] DELETE /identity/v3/users/2e713a1fd64244b9a772dbc208c995a2 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:57:19.680526 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:19.890316 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-59df713e-efaf-4eee-b660-08145fa872ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73bb97aa631a4a09ae9086da1d1c085d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.955207 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-59df713e-efaf-4eee-b660-08145fa872ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73bb97aa631a4a09ae9086da1d1c085d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.955829 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-59df713e-efaf-4eee-b660-08145fa872ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73bb97aa631a4a09ae9086da1d1c085d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.963333 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-59df713e-efaf-4eee-b660-08145fa872ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73bb97aa631a4a09ae9086da1d1c085d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.975875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-59df713e-efaf-4eee-b660-08145fa872ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73bb97aa631a4a09ae9086da1d1c085d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:19.985543 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1332/2693] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:19 2022] DELETE /identity/v3/users/73bb97aa631a4a09ae9086da1d1c085d => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:57:20.071141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:20.072815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:20.140800 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e443e-de09-411e-a08c-dd41d47ceadd admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:20.145687 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:20.197279 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:20.197953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.198229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.198501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] PATH_INFO: `/v3/projects/5c3a94f3e1d14c58a406f9316b0f3af8` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.200146 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c3a94f3e1d14c58a406f9316b0f3af8)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:20.201670 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:20.287158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c3a94f3e1d14c58a406f9316b0f3af8', 'request_id': 'req-7ab3909c-1648-444e-b215-867efb2cc054'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:20.293832 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c3a94f3e1d14c58a406f9316b0f3af8', 'request_id': 'req-7ab3909c-1648-444e-b215-867efb2cc054'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:20.301862 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c3a94f3e1d14c58a406f9316b0f3af8', 'request_id': 'req-7ab3909c-1648-444e-b215-867efb2cc054'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:20.367898 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] The token cache is being invalidate because project 5c3a94f3e1d14c58a406f9316b0f3af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:20.368107 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:20.368704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:20.370231 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1362/2694] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:20 2022] DELETE /identity/v3/projects/5c3a94f3e1d14c58a406f9316b0f3af8 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:20.375077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4388764-c0e7-4616-bbff-1cb4cd510fb9 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.375560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4388764-c0e7-4616-bbff-1cb4cd510fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.375660 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4388764-c0e7-4616-bbff-1cb4cd510fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.396856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e4388764-c0e7-4616-bbff-1cb4cd510fb9 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:20.398041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e4388764-c0e7-4616-bbff-1cb4cd510fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:20.448384 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:20.450074 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:20.472590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1333/2695] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:20.484466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05896e92-d7ea-4698-b0c6-06595a6f751e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.484970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05896e92-d7ea-4698-b0c6-06595a6f751e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.485569 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05896e92-d7ea-4698-b0c6-06595a6f751e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.509417 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-05896e92-d7ea-4698-b0c6-06595a6f751e None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:20.514916 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-05896e92-d7ea-4698-b0c6-06595a6f751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:20.519854 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1334/2696] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:20.524007 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05896e92-d7ea-4698-b0c6-06595a6f751e None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:20.533208 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05896e92-d7ea-4698-b0c6-06595a6f751e None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:20.538185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd117cc-c74b-49ec-938c-241b34efb656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:20.539770 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd117cc-c74b-49ec-938c-241b34efb656 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.540101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd117cc-c74b-49ec-938c-241b34efb656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.540379 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd117cc-c74b-49ec-938c-241b34efb656 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.541081 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd117cc-c74b-49ec-938c-241b34efb656 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:57:20.541386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd117cc-c74b-49ec-938c-241b34efb656 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:20.543099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd117cc-c74b-49ec-938c-241b34efb656 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:20.544001 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3909c-1648-444e-b215-867efb2cc054 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:20.548751 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:20.549382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.549694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.549971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc admin admin] PATH_INFO: `/v3/projects/dd6bca51fdaf415a82632d1f342f2da0` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.551346 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd6bca51fdaf415a82632d1f342f2da0)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:20.561326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:20.562487 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1335/2697] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:57:20.564658 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd117cc-c74b-49ec-938c-241b34efb656 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:20.575612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd117cc-c74b-49ec-938c-241b34efb656 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:20.581813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8befc6-4fdd-4475-8d87-9eb307914c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:20.582462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8befc6-4fdd-4475-8d87-9eb307914c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.582741 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8befc6-4fdd-4475-8d87-9eb307914c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.583036 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8befc6-4fdd-4475-8d87-9eb307914c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.583657 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8befc6-4fdd-4475-8d87-9eb307914c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:20.585588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8befc6-4fdd-4475-8d87-9eb307914c3a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:20.663840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:20.670422 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd6bca51fdaf415a82632d1f342f2da0', 'request_id': 'req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:20.675920 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd6bca51fdaf415a82632d1f342f2da0', 'request_id': 'req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:20.682098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd6bca51fdaf415a82632d1f342f2da0', 'request_id': 'req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:20.695937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1336/2698] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:57:20 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:57:20.699019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8befc6-4fdd-4475-8d87-9eb307914c3a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:20.756733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:20.762596 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8befc6-4fdd-4475-8d87-9eb307914c3a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:20.767209 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405eac7-705d-4d9c-a013-b7f06e106006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:20.767807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405eac7-705d-4d9c-a013-b7f06e106006 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.768087 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405eac7-705d-4d9c-a013-b7f06e106006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.768356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405eac7-705d-4d9c-a013-b7f06e106006 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.768974 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405eac7-705d-4d9c-a013-b7f06e106006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:20.770968 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405eac7-705d-4d9c-a013-b7f06e106006 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:20.779327 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc admin admin] The token cache is being invalidate because project dd6bca51fdaf415a82632d1f342f2da0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:20.779836 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:20.785595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-7b79926b-8f73-4f77-b84b-e38f3ecc01fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:20.785595 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1363/2699] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:20 2022] DELETE /identity/v3/projects/dd6bca51fdaf415a82632d1f342f2da0 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:20.787142 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6350628-6aca-421d-aca7-d1830b32581b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.787425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6350628-6aca-421d-aca7-d1830b32581b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.787705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6350628-6aca-421d-aca7-d1830b32581b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.807851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-d6350628-6aca-421d-aca7-d1830b32581b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:20.809051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d6350628-6aca-421d-aca7-d1830b32581b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:20.817491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1364/2700] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:20.824921 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cc78e-bb01-4e10-a9b7-aba539123683 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.825210 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cc78e-bb01-4e10-a9b7-aba539123683 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.825553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cc78e-bb01-4e10-a9b7-aba539123683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.847930 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-4b4cc78e-bb01-4e10-a9b7-aba539123683 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:20.849137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-4b4cc78e-bb01-4e10-a9b7-aba539123683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:20.857994 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1365/2701] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:57:20.862125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cc78e-bb01-4e10-a9b7-aba539123683 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:20.871248 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cc78e-bb01-4e10-a9b7-aba539123683 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:20.876232 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d12d1c-bdc4-42de-b782-7147bdd6f90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:20.876851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d12d1c-bdc4-42de-b782-7147bdd6f90d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.877158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d12d1c-bdc4-42de-b782-7147bdd6f90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.877427 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d12d1c-bdc4-42de-b782-7147bdd6f90d admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.878207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d12d1c-bdc4-42de-b782-7147bdd6f90d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:57:20.878514 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d12d1c-bdc4-42de-b782-7147bdd6f90d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:20.880164 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d12d1c-bdc4-42de-b782-7147bdd6f90d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:20.881552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1337/2702] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:57:20 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:57:20.885309 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405eac7-705d-4d9c-a013-b7f06e106006 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:20.887075 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6405eac7-705d-4d9c-a013-b7f06e106006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:20.895929 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1366/2703] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:57:20.897171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d12d1c-bdc4-42de-b782-7147bdd6f90d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:20.900886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405eac7-705d-4d9c-a013-b7f06e106006 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:20.905807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ff808-5e5f-4c6c-bf03-e409e905881e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:20.906405 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ff808-5e5f-4c6c-bf03-e409e905881e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.906682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ff808-5e5f-4c6c-bf03-e409e905881e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.906952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ff808-5e5f-4c6c-bf03-e409e905881e admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.907559 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d12d1c-bdc4-42de-b782-7147bdd6f90d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:20.907707 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ff808-5e5f-4c6c-bf03-e409e905881e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:20.909322 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ff808-5e5f-4c6c-bf03-e409e905881e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:20.913012 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d033a91-7a96-4041-aab7-c65eba0d9a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:20.913705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d033a91-7a96-4041-aab7-c65eba0d9a73 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.914755 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d033a91-7a96-4041-aab7-c65eba0d9a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.914965 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d033a91-7a96-4041-aab7-c65eba0d9a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.915723 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d033a91-7a96-4041-aab7-c65eba0d9a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:20.917705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d033a91-7a96-4041-aab7-c65eba0d9a73 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:20.920282 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1338/2704] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:57:20.924147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ff808-5e5f-4c6c-bf03-e409e905881e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:20.934733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ff808-5e5f-4c6c-bf03-e409e905881e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:20.939882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00aab66-2873-4a37-80a0-c2a4ded3772f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:20.940994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00aab66-2873-4a37-80a0-c2a4ded3772f admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.940994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00aab66-2873-4a37-80a0-c2a4ded3772f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.940994 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00aab66-2873-4a37-80a0-c2a4ded3772f admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.941695 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00aab66-2873-4a37-80a0-c2a4ded3772f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:20.943279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00aab66-2873-4a37-80a0-c2a4ded3772f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:20.955771 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1339/2705] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:57:20.959921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00aab66-2873-4a37-80a0-c2a4ded3772f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:20.970289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00aab66-2873-4a37-80a0-c2a4ded3772f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:20.975104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f6211-aa78-4559-9939-6c9cae9c9dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:20.975770 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f6211-aa78-4559-9939-6c9cae9c9dec admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:20.976053 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f6211-aa78-4559-9939-6c9cae9c9dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:20.976324 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f6211-aa78-4559-9939-6c9cae9c9dec admin admin] PATH_INFO: `/v3/projects/80af7ca4d144441a9662afa5ce04b8a1/users/83e9595f4e25446890e490d80f4954ed/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:20.990985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f6211-aa78-4559-9939-6c9cae9c9dec admin admin] RBAC: Authorizing `identity:create_grant(project_id=80af7ca4d144441a9662afa5ce04b8a1, user_id=83e9595f4e25446890e490d80f4954ed, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:20.993843 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f6211-aa78-4559-9939-6c9cae9c9dec admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:21.024061 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1367/2706] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:57:20 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:21.028299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d033a91-7a96-4041-aab7-c65eba0d9a73 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:21.093761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1340/2707] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:20 2022] PUT /identity/v3/projects/80af7ca4d144441a9662afa5ce04b8a1/users/83e9595f4e25446890e490d80f4954ed/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:57:21.099343 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d033a91-7a96-4041-aab7-c65eba0d9a73 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:21.104595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb7183-a169-4e64-be92-f7ab6426e4c5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.105108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb7183-a169-4e64-be92-f7ab6426e4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.105274 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb7183-a169-4e64-be92-f7ab6426e4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.142130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-eceb7183-a169-4e64-be92-f7ab6426e4c5 None None] MFA Rules not processed for user `83e9595f4e25446890e490d80f4954ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:21.143617 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-eceb7183-a169-4e64-be92-f7ab6426e4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:21.173238 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f386da0-e294-4212-b176-ac5b515f2432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:21.173925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f386da0-e294-4212-b176-ac5b515f2432 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.174204 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f386da0-e294-4212-b176-ac5b515f2432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.174470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f386da0-e294-4212-b176-ac5b515f2432 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.175094 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f386da0-e294-4212-b176-ac5b515f2432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:21.177048 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f386da0-e294-4212-b176-ac5b515f2432 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:21.217563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1341/2708] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:21.225474 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb7183-a169-4e64-be92-f7ab6426e4c5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:21.236366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb7183-a169-4e64-be92-f7ab6426e4c5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:21.241045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36629367-9aae-4349-9c40-918a46ac331c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:21.241698 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36629367-9aae-4349-9c40-918a46ac331c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.247381 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36629367-9aae-4349-9c40-918a46ac331c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.247381 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36629367-9aae-4349-9c40-918a46ac331c admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.247381 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36629367-9aae-4349-9c40-918a46ac331c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:21.247381 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36629367-9aae-4349-9c40-918a46ac331c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:21.271017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1368/2709] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:57:21 2022] POST /identity/v3/users => generated 287 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:57:21.276240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f386da0-e294-4212-b176-ac5b515f2432 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:21.286533 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f386da0-e294-4212-b176-ac5b515f2432 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:21.291571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82294a-972b-4aa4-8e9b-b1daf2a47611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:21.292173 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82294a-972b-4aa4-8e9b-b1daf2a47611 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.292479 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82294a-972b-4aa4-8e9b-b1daf2a47611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.292752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82294a-972b-4aa4-8e9b-b1daf2a47611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.293423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82294a-972b-4aa4-8e9b-b1daf2a47611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:21.295063 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82294a-972b-4aa4-8e9b-b1daf2a47611 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:21.305115 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1369/2710] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:57:21.309155 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82294a-972b-4aa4-8e9b-b1daf2a47611 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:21.319075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82294a-972b-4aa4-8e9b-b1daf2a47611 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:21.323666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1342/2711] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:57:21 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:57:21.329782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b4a7a-0e46-4f1f-bb36-2615e7b6b95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:21.329782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b4a7a-0e46-4f1f-bb36-2615e7b6b95a admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.329782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b4a7a-0e46-4f1f-bb36-2615e7b6b95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.329782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b4a7a-0e46-4f1f-bb36-2615e7b6b95a admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.330171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b4a7a-0e46-4f1f-bb36-2615e7b6b95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:21.330171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b4a7a-0e46-4f1f-bb36-2615e7b6b95a admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:21.330171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:21.330850 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36629367-9aae-4349-9c40-918a46ac331c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:21.341319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1370/2712] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:57:21.348240 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b4a7a-0e46-4f1f-bb36-2615e7b6b95a admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:21.358310 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b4a7a-0e46-4f1f-bb36-2615e7b6b95a admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:21.397272 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36629367-9aae-4349-9c40-918a46ac331c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:21.403237 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2bc39-6b43-493a-9ace-f87d1ab52c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:21.403838 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2bc39-6b43-493a-9ace-f87d1ab52c71 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.404111 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2bc39-6b43-493a-9ace-f87d1ab52c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.404382 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2bc39-6b43-493a-9ace-f87d1ab52c71 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.405000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2bc39-6b43-493a-9ace-f87d1ab52c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:21.406931 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2bc39-6b43-493a-9ace-f87d1ab52c71 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:21.416153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bab20-9f2f-4ef8-a71e-89c29c8d8d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:21.416877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bab20-9f2f-4ef8-a71e-89c29c8d8d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.417183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bab20-9f2f-4ef8-a71e-89c29c8d8d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.419594 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bab20-9f2f-4ef8-a71e-89c29c8d8d38 admin admin] PATH_INFO: `/v3/projects/8b6338088b03415aa67ca34647ca0700/users/debdf689e44c48e1bca1c7ad37f8335f/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.419719 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:21.434704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bab20-9f2f-4ef8-a71e-89c29c8d8d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b6338088b03415aa67ca34647ca0700, user_id=debdf689e44c48e1bca1c7ad37f8335f, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:21.437827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bab20-9f2f-4ef8-a71e-89c29c8d8d38 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:21.527934 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1343/2713] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:57:21 2022] POST /identity/v3/users => generated 299 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:21.531099 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2bc39-6b43-493a-9ace-f87d1ab52c71 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:21.533848 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1371/2714] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:21 2022] PUT /identity/v3/projects/8b6338088b03415aa67ca34647ca0700/users/debdf689e44c48e1bca1c7ad37f8335f/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:57:21.538573 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a08fb-b7ec-48e2-bc1b-3e16652c10d5 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.538860 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a08fb-b7ec-48e2-bc1b-3e16652c10d5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.539197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a08fb-b7ec-48e2-bc1b-3e16652c10d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.542041 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2bc39-6b43-493a-9ace-f87d1ab52c71 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:21.558314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e42a08fb-b7ec-48e2-bc1b-3e16652c10d5 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:21.559513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e42a08fb-b7ec-48e2-bc1b-3e16652c10d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:21.608210 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbc27c-705b-46ef-8ecf-0b701504f40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:21.608881 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbc27c-705b-46ef-8ecf-0b701504f40d admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.609129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbc27c-705b-46ef-8ecf-0b701504f40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.609767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbc27c-705b-46ef-8ecf-0b701504f40d admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.610042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbc27c-705b-46ef-8ecf-0b701504f40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:21.611898 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbc27c-705b-46ef-8ecf-0b701504f40d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:21.623954 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1344/2715] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:57:21.628845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbc27c-705b-46ef-8ecf-0b701504f40d admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:21.640410 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1372/2716] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:21.641955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbc27c-705b-46ef-8ecf-0b701504f40d admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:21.649092 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bb8c7-80af-40bf-860f-b5068bc86817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:21.649800 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bb8c7-80af-40bf-860f-b5068bc86817 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.650084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bb8c7-80af-40bf-860f-b5068bc86817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.650275 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a08fb-b7ec-48e2-bc1b-3e16652c10d5 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:21.650388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bb8c7-80af-40bf-860f-b5068bc86817 admin admin] PATH_INFO: `/v3/projects/acbf1fe420134365bea811988112fe52/users/7801a7f4375643abbd7d4984d7b112d0/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.651776 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e42a08fb-b7ec-48e2-bc1b-3e16652c10d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:21.668275 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892bb8c7-80af-40bf-860f-b5068bc86817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acbf1fe420134365bea811988112fe52, user_id=7801a7f4375643abbd7d4984d7b112d0, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:21.671225 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892bb8c7-80af-40bf-860f-b5068bc86817 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:21.713967 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a08fb-b7ec-48e2-bc1b-3e16652c10d5 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:21.717508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b24399-20e4-460e-8060-d0f1b4114f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:21.718144 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b24399-20e4-460e-8060-d0f1b4114f65 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.718426 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b24399-20e4-460e-8060-d0f1b4114f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.718702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b24399-20e4-460e-8060-d0f1b4114f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.728511 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b24399-20e4-460e-8060-d0f1b4114f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:21.730033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b24399-20e4-460e-8060-d0f1b4114f65 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:21.739737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1373/2717] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:57:21.761539 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1345/2718] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:21 2022] PUT /identity/v3/projects/acbf1fe420134365bea811988112fe52/users/7801a7f4375643abbd7d4984d7b112d0/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:21.765679 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b24399-20e4-460e-8060-d0f1b4114f65 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:21.775432 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b24399-20e4-460e-8060-d0f1b4114f65 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:21.794387 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:21.833562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14d59f-ab40-4840-9f71-f95c319a3c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:21.834171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14d59f-ab40-4840-9f71-f95c319a3c64 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.834450 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14d59f-ab40-4840-9f71-f95c319a3c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.834724 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14d59f-ab40-4840-9f71-f95c319a3c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.835371 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14d59f-ab40-4840-9f71-f95c319a3c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:21.837017 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14d59f-ab40-4840-9f71-f95c319a3c64 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:21.850588 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1374/2719] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:21.851987 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bb8c7-80af-40bf-860f-b5068bc86817 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:21.862675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bb8c7-80af-40bf-860f-b5068bc86817 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:21.867407 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4760e2-da40-4dff-ac34-cf50577454f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:21.868776 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4760e2-da40-4dff-ac34-cf50577454f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.869165 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4760e2-da40-4dff-ac34-cf50577454f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.869445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4760e2-da40-4dff-ac34-cf50577454f4 admin admin] PATH_INFO: `/v3/projects/acbf1fe420134365bea811988112fe52/users/7801a7f4375643abbd7d4984d7b112d0/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.871400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4760e2-da40-4dff-ac34-cf50577454f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acbf1fe420134365bea811988112fe52, user_id=7801a7f4375643abbd7d4984d7b112d0, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:21.875142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4760e2-da40-4dff-ac34-cf50577454f4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:21.885706 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1346/2720] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:21 2022] PUT /identity/v3/projects/acbf1fe420134365bea811988112fe52/users/7801a7f4375643abbd7d4984d7b112d0/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:57:21.897399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1a282-4bbe-4718-95cc-6dd7fada2e56 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:21.897732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1a282-4bbe-4718-95cc-6dd7fada2e56 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:21.898013 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1a282-4bbe-4718-95cc-6dd7fada2e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:21.944251 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-fce1a282-4bbe-4718-95cc-6dd7fada2e56 None None] MFA Rules not processed for user `7801a7f4375643abbd7d4984d7b112d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:21.945755 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fce1a282-4bbe-4718-95cc-6dd7fada2e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:22.018826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1375/2721] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:22.030193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4760e2-da40-4dff-ac34-cf50577454f4 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:22.031920 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3d4760e2-da40-4dff-ac34-cf50577454f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:22.116049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4760e2-da40-4dff-ac34-cf50577454f4 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:22.120024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a6aae-b455-4723-a07a-64658c4d9654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:22.120639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a6aae-b455-4723-a07a-64658c4d9654 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:22.126019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a6aae-b455-4723-a07a-64658c4d9654 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:22.126019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a6aae-b455-4723-a07a-64658c4d9654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:22.132347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a6aae-b455-4723-a07a-64658c4d9654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:22.133913 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a6aae-b455-4723-a07a-64658c4d9654 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:22.146338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1347/2722] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:22.218621 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1a282-4bbe-4718-95cc-6dd7fada2e56 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:22.288641 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1a282-4bbe-4718-95cc-6dd7fada2e56 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:22.294474 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:22.295292 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:22.295571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:22.295844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] PATH_INFO: `/v3/users/83e9595f4e25446890e490d80f4954ed` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:22.297076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e9595f4e25446890e490d80f4954ed)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:22.299063 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:22.552293 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:22.554392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e9595f4e25446890e490d80f4954ed'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:22.640569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e9595f4e25446890e490d80f4954ed'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:22.641287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e9595f4e25446890e490d80f4954ed'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:22.649267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e9595f4e25446890e490d80f4954ed'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:22.660657 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e9595f4e25446890e490d80f4954ed'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:22.666703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1376/2723] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:22 2022] DELETE /identity/v3/users/83e9595f4e25446890e490d80f4954ed => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:22.671342 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485599d9-4913-4935-96e3-e367808dc909 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:22.671894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485599d9-4913-4935-96e3-e367808dc909 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:22.671894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485599d9-4913-4935-96e3-e367808dc909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:22.690789 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-485599d9-4913-4935-96e3-e367808dc909 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:22.691985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-485599d9-4913-4935-96e3-e367808dc909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:22.736369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:22.754523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1348/2724] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:22.764973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:22.766411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:22.827486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570fa1b-f983-401d-84e7-4c585ac566e7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:22.831318 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928e128-1bdb-42ae-9e45-4cc5202a39b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:22.831941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928e128-1bdb-42ae-9e45-4cc5202a39b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:22.832218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928e128-1bdb-42ae-9e45-4cc5202a39b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:22.832642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928e128-1bdb-42ae-9e45-4cc5202a39b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:22.842655 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928e128-1bdb-42ae-9e45-4cc5202a39b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:22.844112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928e128-1bdb-42ae-9e45-4cc5202a39b2 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:22.854217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1377/2725] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:23.226193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f1b18-558b-4ad3-9d76-e151327174d7 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:23.227156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f1b18-558b-4ad3-9d76-e151327174d7 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:23.227528 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f1b18-558b-4ad3-9d76-e151327174d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:23.264756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9b5f1b18-558b-4ad3-9d76-e151327174d7 None None] MFA Rules not processed for user `debdf689e44c48e1bca1c7ad37f8335f`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:23.267079 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9b5f1b18-558b-4ad3-9d76-e151327174d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:23.341302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1349/2726] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:23.352611 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928e128-1bdb-42ae-9e45-4cc5202a39b2 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:23.354034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b928e128-1bdb-42ae-9e45-4cc5202a39b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:23.387496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f1b18-558b-4ad3-9d76-e151327174d7 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:23.388638 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9b5f1b18-558b-4ad3-9d76-e151327174d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:23.408858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f1b18-558b-4ad3-9d76-e151327174d7 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:23.414980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:23.415618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:23.415896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:23.416167 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] PATH_INFO: `/v3/projects/80af7ca4d144441a9662afa5ce04b8a1` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:23.417399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80af7ca4d144441a9662afa5ce04b8a1)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:23.419447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:23.420733 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928e128-1bdb-42ae-9e45-4cc5202a39b2 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:23.424980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0b861-3a1f-43fe-ae43-e5d7b3f42de0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:23.425618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0b861-3a1f-43fe-ae43-e5d7b3f42de0 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:23.425923 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0b861-3a1f-43fe-ae43-e5d7b3f42de0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:23.426176 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0b861-3a1f-43fe-ae43-e5d7b3f42de0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:23.436662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0b861-3a1f-43fe-ae43-e5d7b3f42de0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:23.438524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0b861-3a1f-43fe-ae43-e5d7b3f42de0 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:23.448742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1378/2727] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:23.529844 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80af7ca4d144441a9662afa5ce04b8a1', 'request_id': 'req-611b0514-14de-4ce1-9884-f94425e95160'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:23.536006 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80af7ca4d144441a9662afa5ce04b8a1', 'request_id': 'req-611b0514-14de-4ce1-9884-f94425e95160'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:23.542223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80af7ca4d144441a9662afa5ce04b8a1', 'request_id': 'req-611b0514-14de-4ce1-9884-f94425e95160'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:23.600390 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] The token cache is being invalidate because project 80af7ca4d144441a9662afa5ce04b8a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:23.600595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:23.601231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:23.604572 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1350/2728] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:23 2022] DELETE /identity/v3/projects/80af7ca4d144441a9662afa5ce04b8a1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:57:23.608667 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0b861-3a1f-43fe-ae43-e5d7b3f42de0 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:23.619709 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0b861-3a1f-43fe-ae43-e5d7b3f42de0 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:23.675838 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ab260-b05c-4a08-b11c-2c558a783486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:23.676460 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ab260-b05c-4a08-b11c-2c558a783486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:23.676740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ab260-b05c-4a08-b11c-2c558a783486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:23.677016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ab260-b05c-4a08-b11c-2c558a783486 admin admin] PATH_INFO: `/v3/users/7801a7f4375643abbd7d4984d7b112d0` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:23.678153 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ab260-b05c-4a08-b11c-2c558a783486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7801a7f4375643abbd7d4984d7b112d0)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:23.680149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ab260-b05c-4a08-b11c-2c558a783486 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:23.691302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:23.862403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1c5ab260-b05c-4a08-b11c-2c558a783486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7801a7f4375643abbd7d4984d7b112d0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:23.949296 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1c5ab260-b05c-4a08-b11c-2c558a783486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7801a7f4375643abbd7d4984d7b112d0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:23.950310 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1c5ab260-b05c-4a08-b11c-2c558a783486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7801a7f4375643abbd7d4984d7b112d0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:23.959309 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1c5ab260-b05c-4a08-b11c-2c558a783486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7801a7f4375643abbd7d4984d7b112d0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:23.972737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-1c5ab260-b05c-4a08-b11c-2c558a783486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7801a7f4375643abbd7d4984d7b112d0'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:23.979085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1379/2729] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:23 2022] DELETE /identity/v3/users/7801a7f4375643abbd7d4984d7b112d0 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:24.449088 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:24.450686 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:24.465177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b0514-14de-4ce1-9884-f94425e95160 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:24.522231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f5e59-765d-4735-bfa5-df5270da2013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:24.522862 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f5e59-765d-4735-bfa5-df5270da2013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:24.523143 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f5e59-765d-4735-bfa5-df5270da2013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:24.523416 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f5e59-765d-4735-bfa5-df5270da2013 admin admin] PATH_INFO: `/v3/projects/acbf1fe420134365bea811988112fe52` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:24.525033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f5e59-765d-4735-bfa5-df5270da2013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acbf1fe420134365bea811988112fe52)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:24.526961 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f5e59-765d-4735-bfa5-df5270da2013 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:24.528506 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:24.636156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0c8f5e59-765d-4735-bfa5-df5270da2013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acbf1fe420134365bea811988112fe52', 'request_id': 'req-0c8f5e59-765d-4735-bfa5-df5270da2013'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:24.642349 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0c8f5e59-765d-4735-bfa5-df5270da2013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acbf1fe420134365bea811988112fe52', 'request_id': 'req-0c8f5e59-765d-4735-bfa5-df5270da2013'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:24.647719 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0c8f5e59-765d-4735-bfa5-df5270da2013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acbf1fe420134365bea811988112fe52', 'request_id': 'req-0c8f5e59-765d-4735-bfa5-df5270da2013'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:24.715312 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0c8f5e59-765d-4735-bfa5-df5270da2013 admin admin] The token cache is being invalidate because project acbf1fe420134365bea811988112fe52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:24.715497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0c8f5e59-765d-4735-bfa5-df5270da2013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:24.716177 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-0c8f5e59-765d-4735-bfa5-df5270da2013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:24.717697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1351/2730] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:24 2022] DELETE /identity/v3/projects/acbf1fe420134365bea811988112fe52 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:24.722915 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7dc58-dee6-4eb0-a970-b53fc9135a1b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:24.723202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7dc58-dee6-4eb0-a970-b53fc9135a1b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:24.723483 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7dc58-dee6-4eb0-a970-b53fc9135a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:24.743209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-52b7dc58-dee6-4eb0-a970-b53fc9135a1b None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:24.744411 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-52b7dc58-dee6-4eb0-a970-b53fc9135a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:24.805154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1380/2731] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:24.813473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83bf8f-b2a1-4aae-9a89-4ba0fab62c93 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:24.813790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83bf8f-b2a1-4aae-9a89-4ba0fab62c93 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:24.814069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83bf8f-b2a1-4aae-9a89-4ba0fab62c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:24.834273 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4a83bf8f-b2a1-4aae-9a89-4ba0fab62c93 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:24.835480 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4a83bf8f-b2a1-4aae-9a89-4ba0fab62c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:24.844084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1352/2732] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:24.848519 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7dc58-dee6-4eb0-a970-b53fc9135a1b None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:24.859086 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7dc58-dee6-4eb0-a970-b53fc9135a1b None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:24.864015 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3329c65-183f-4824-9a93-fdc1b989d868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:24.864631 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3329c65-183f-4824-9a93-fdc1b989d868 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:24.864872 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3329c65-183f-4824-9a93-fdc1b989d868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:24.865139 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3329c65-183f-4824-9a93-fdc1b989d868 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:24.871103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3329c65-183f-4824-9a93-fdc1b989d868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:57:24.871103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3329c65-183f-4824-9a93-fdc1b989d868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:24.871103 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3329c65-183f-4824-9a93-fdc1b989d868 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:24.880592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1381/2733] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:57:24.885077 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83bf8f-b2a1-4aae-9a89-4ba0fab62c93 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:24.894394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83bf8f-b2a1-4aae-9a89-4ba0fab62c93 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:24.899178 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b8c58-c1d0-4b71-a856-94d50670904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:24.899767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b8c58-c1d0-4b71-a856-94d50670904b admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:24.900048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b8c58-c1d0-4b71-a856-94d50670904b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:24.900323 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b8c58-c1d0-4b71-a856-94d50670904b admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:24.900942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b8c58-c1d0-4b71-a856-94d50670904b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:24.902875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b8c58-c1d0-4b71-a856-94d50670904b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:25.016119 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1353/2734] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:57:24 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:25.019649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3329c65-183f-4824-9a93-fdc1b989d868 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:25.030803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3329c65-183f-4824-9a93-fdc1b989d868 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:25.092558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c38b7c-7bee-45ba-be0a-746402533ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:25.093257 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c38b7c-7bee-45ba-be0a-746402533ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:25.093471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c38b7c-7bee-45ba-be0a-746402533ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:25.093741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c38b7c-7bee-45ba-be0a-746402533ef9 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:25.094381 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c38b7c-7bee-45ba-be0a-746402533ef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:25.096306 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c38b7c-7bee-45ba-be0a-746402533ef9 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:25.179020 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1382/2735] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:57:25 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:25.183817 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b8c58-c1d0-4b71-a856-94d50670904b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:25.194545 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b8c58-c1d0-4b71-a856-94d50670904b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:25.201433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae31c29-76dc-4626-8786-84d3ae186b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:25.206016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae31c29-76dc-4626-8786-84d3ae186b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:25.206016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae31c29-76dc-4626-8786-84d3ae186b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:25.206016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae31c29-76dc-4626-8786-84d3ae186b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:25.206016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae31c29-76dc-4626-8786-84d3ae186b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:25.206016 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae31c29-76dc-4626-8786-84d3ae186b13 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:25.216340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1354/2736] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:25.220618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c38b7c-7bee-45ba-be0a-746402533ef9 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:25.227409 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:25.231258 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c38b7c-7bee-45ba-be0a-746402533ef9 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:25.235940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a166ea-2c26-4653-931a-df391c6707d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:25.250154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a166ea-2c26-4653-931a-df391c6707d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:25.250154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a166ea-2c26-4653-931a-df391c6707d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:25.250154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a166ea-2c26-4653-931a-df391c6707d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:25.250154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a166ea-2c26-4653-931a-df391c6707d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:25.250154 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a166ea-2c26-4653-931a-df391c6707d1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:25.250734 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1383/2737] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:25.255142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae31c29-76dc-4626-8786-84d3ae186b13 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:25.266021 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae31c29-76dc-4626-8786-84d3ae186b13 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:25.272104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14507706-c298-46d4-bc74-fca9eeda4230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:25.272104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14507706-c298-46d4-bc74-fca9eeda4230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:25.272104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14507706-c298-46d4-bc74-fca9eeda4230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:25.272104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14507706-c298-46d4-bc74-fca9eeda4230 admin admin] PATH_INFO: `/v3/projects/d7271f1c9468434e8e6bcb51b6aad1e7/users/485fb47b58464356b3a703bb44363952/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:25.287262 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14507706-c298-46d4-bc74-fca9eeda4230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7271f1c9468434e8e6bcb51b6aad1e7, user_id=485fb47b58464356b3a703bb44363952, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:25.289985 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14507706-c298-46d4-bc74-fca9eeda4230 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:25.373991 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1355/2738] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:25 2022] PUT /identity/v3/projects/d7271f1c9468434e8e6bcb51b6aad1e7/users/485fb47b58464356b3a703bb44363952/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:25.385464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a965835-bbd5-4fdd-a827-2a004400564c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:25.385869 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a965835-bbd5-4fdd-a827-2a004400564c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:25.386246 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a965835-bbd5-4fdd-a827-2a004400564c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:25.414332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5a965835-bbd5-4fdd-a827-2a004400564c None None] MFA Rules not processed for user `485fb47b58464356b3a703bb44363952`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:25.415560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5a965835-bbd5-4fdd-a827-2a004400564c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:25.485223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1384/2739] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:25.489892 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14507706-c298-46d4-bc74-fca9eeda4230 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:25.563527 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14507706-c298-46d4-bc74-fca9eeda4230 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:25.568265 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653130f-fd1c-4bdc-93a4-a6ec14c527e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:25.568868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653130f-fd1c-4bdc-93a4-a6ec14c527e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:25.569166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653130f-fd1c-4bdc-93a4-a6ec14c527e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:25.569423 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653130f-fd1c-4bdc-93a4-a6ec14c527e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:25.570080 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653130f-fd1c-4bdc-93a4-a6ec14c527e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:25.572046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653130f-fd1c-4bdc-93a4-a6ec14c527e4 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:25.669579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1356/2740] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:57:25 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:25.672548 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a965835-bbd5-4fdd-a827-2a004400564c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:25.684455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a965835-bbd5-4fdd-a827-2a004400564c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:25.745888 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1380fa-6586-478f-8c50-ecc208a3077f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:25.749386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1380fa-6586-478f-8c50-ecc208a3077f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:25.749386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1380fa-6586-478f-8c50-ecc208a3077f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:25.749386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1380fa-6586-478f-8c50-ecc208a3077f admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:25.749386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1380fa-6586-478f-8c50-ecc208a3077f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:25.749953 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1380fa-6586-478f-8c50-ecc208a3077f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:25.831199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1385/2741] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:57:25 2022] POST /identity/v3/users => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:25.835585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653130f-fd1c-4bdc-93a4-a6ec14c527e4 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:25.846712 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653130f-fd1c-4bdc-93a4-a6ec14c527e4 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:25.851416 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f43a8-2ccc-4d6f-81bb-42e028285c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:25.852633 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f43a8-2ccc-4d6f-81bb-42e028285c8a admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:25.852914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f43a8-2ccc-4d6f-81bb-42e028285c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:25.853190 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f43a8-2ccc-4d6f-81bb-42e028285c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:25.853921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f43a8-2ccc-4d6f-81bb-42e028285c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:25.855478 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f43a8-2ccc-4d6f-81bb-42e028285c8a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:25.860766 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:25.866487 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1357/2742] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:25.870893 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1380fa-6586-478f-8c50-ecc208a3077f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:25.882189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1380fa-6586-478f-8c50-ecc208a3077f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:25.887189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4424878-d478-4d74-a616-38b5b90dd6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:25.887983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4424878-d478-4d74-a616-38b5b90dd6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:25.888311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4424878-d478-4d74-a616-38b5b90dd6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:25.888634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4424878-d478-4d74-a616-38b5b90dd6e4 admin admin] PATH_INFO: `/v3/projects/abeb4eee85484282963aba4615f6c291/users/442c531ed5eb425390402393a2a6fc37/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:25.906911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4424878-d478-4d74-a616-38b5b90dd6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abeb4eee85484282963aba4615f6c291, user_id=442c531ed5eb425390402393a2a6fc37, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:25.909870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4424878-d478-4d74-a616-38b5b90dd6e4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:25.984897 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1386/2743] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:25 2022] PUT /identity/v3/projects/abeb4eee85484282963aba4615f6c291/users/442c531ed5eb425390402393a2a6fc37/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:25.989287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f43a8-2ccc-4d6f-81bb-42e028285c8a admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:25.999768 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f43a8-2ccc-4d6f-81bb-42e028285c8a admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:26.060399 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d36af-006c-474b-b696-ff9689af1c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:26.061164 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d36af-006c-474b-b696-ff9689af1c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:26.061554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d36af-006c-474b-b696-ff9689af1c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:26.061984 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d36af-006c-474b-b696-ff9689af1c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:26.063025 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d36af-006c-474b-b696-ff9689af1c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:26.064819 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d36af-006c-474b-b696-ff9689af1c5a admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:26.079902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1358/2744] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:57:26.084194 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4424878-d478-4d74-a616-38b5b90dd6e4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:26.095690 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4424878-d478-4d74-a616-38b5b90dd6e4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:26.102794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955f9b7-79c4-4993-bc00-79b229c0d761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:26.103489 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955f9b7-79c4-4993-bc00-79b229c0d761 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:26.103825 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955f9b7-79c4-4993-bc00-79b229c0d761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:26.104102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955f9b7-79c4-4993-bc00-79b229c0d761 admin admin] PATH_INFO: `/v3/projects/abeb4eee85484282963aba4615f6c291/users/442c531ed5eb425390402393a2a6fc37/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:26.106185 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955f9b7-79c4-4993-bc00-79b229c0d761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abeb4eee85484282963aba4615f6c291, user_id=442c531ed5eb425390402393a2a6fc37, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:26.108926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955f9b7-79c4-4993-bc00-79b229c0d761 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:26.120080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1387/2745] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:26 2022] PUT /identity/v3/projects/abeb4eee85484282963aba4615f6c291/users/442c531ed5eb425390402393a2a6fc37/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:26.131724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb090a-02ef-4221-a792-e7cc508ed0b5 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:26.132001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb090a-02ef-4221-a792-e7cc508ed0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:26.132278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb090a-02ef-4221-a792-e7cc508ed0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:26.168793 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-21bb090a-02ef-4221-a792-e7cc508ed0b5 None None] MFA Rules not processed for user `442c531ed5eb425390402393a2a6fc37`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:26.170332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-21bb090a-02ef-4221-a792-e7cc508ed0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:26.234241 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02266b-4fa7-4dd7-8efe-8233989e13d9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:26.234531 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02266b-4fa7-4dd7-8efe-8233989e13d9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:26.234805 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02266b-4fa7-4dd7-8efe-8233989e13d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:26.261919 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-ee02266b-4fa7-4dd7-8efe-8233989e13d9 None None] MFA Rules not processed for user `debdf689e44c48e1bca1c7ad37f8335f`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:26.263583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ee02266b-4fa7-4dd7-8efe-8233989e13d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:26.294593 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1359/2746] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:26.308149 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:26.310354 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb090a-02ef-4221-a792-e7cc508ed0b5 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:26.312587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-21bb090a-02ef-4221-a792-e7cc508ed0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:26.345012 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1388/2747] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:57:26.395586 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb090a-02ef-4221-a792-e7cc508ed0b5 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:26.400796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb71f51-f63e-4d27-be0c-7157a9b132ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:26.401549 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb71f51-f63e-4d27-be0c-7157a9b132ea service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:26.401893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb71f51-f63e-4d27-be0c-7157a9b132ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:26.402171 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb71f51-f63e-4d27-be0c-7157a9b132ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:26.470159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:26.501803 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb71f51-f63e-4d27-be0c-7157a9b132ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:26.503893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb71f51-f63e-4d27-be0c-7157a9b132ea service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:26.514868 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1360/2748] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:57:28.578158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02266b-4fa7-4dd7-8efe-8233989e13d9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:28.579617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ee02266b-4fa7-4dd7-8efe-8233989e13d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:28.690482 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02266b-4fa7-4dd7-8efe-8233989e13d9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:28.695596 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391738a0-3176-4cc5-86ac-651f86abfd1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0aa6b215f646c280af245a89ccf849', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:28.696196 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391738a0-3176-4cc5-86ac-651f86abfd1c service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:28.696473 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391738a0-3176-4cc5-86ac-651f86abfd1c service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:28.696739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391738a0-3176-4cc5-86ac-651f86abfd1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:28.698604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-391738a0-3176-4cc5-86ac-651f86abfd1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:28.710486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391738a0-3176-4cc5-86ac-651f86abfd1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:28.712470 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391738a0-3176-4cc5-86ac-651f86abfd1c service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:28.723499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1389/2749] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:31.866957 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb71f51-f63e-4d27-be0c-7157a9b132ea service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:31.876433 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb71f51-f63e-4d27-be0c-7157a9b132ea service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:31.880502 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add03e53-87f7-4d7c-b9a3-0b2a0227f375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:31.881970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add03e53-87f7-4d7c-b9a3-0b2a0227f375 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:31.881970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add03e53-87f7-4d7c-b9a3-0b2a0227f375 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:31.881970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add03e53-87f7-4d7c-b9a3-0b2a0227f375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:31.892261 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add03e53-87f7-4d7c-b9a3-0b2a0227f375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:31.893767 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add03e53-87f7-4d7c-b9a3-0b2a0227f375 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:31.903912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1361/2750] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:37.558161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391738a0-3176-4cc5-86ac-651f86abfd1c service placement] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:37.635651 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391738a0-3176-4cc5-86ac-651f86abfd1c service placement] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:37.642532 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:37.643178 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:37.643477 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:37.643721 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] PATH_INFO: `/v3/users/485fb47b58464356b3a703bb44363952` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:37.644851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=485fb47b58464356b3a703bb44363952)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:37.646877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:37.903277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '485fb47b58464356b3a703bb44363952'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:37.907610 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:37.979137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '485fb47b58464356b3a703bb44363952'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:37.979792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '485fb47b58464356b3a703bb44363952'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:37.987464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '485fb47b58464356b3a703bb44363952'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:38.000826 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '485fb47b58464356b3a703bb44363952'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:38.007308 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1390/2751] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:37 2022] DELETE /identity/v3/users/485fb47b58464356b3a703bb44363952 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:38.012072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c43bfc-0305-497e-94a7-3c0080ca7b71 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:38.012601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c43bfc-0305-497e-94a7-3c0080ca7b71 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:38.012601 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c43bfc-0305-497e-94a7-3c0080ca7b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:38.031536 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-07c43bfc-0305-497e-94a7-3c0080ca7b71 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:38.032742 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-07c43bfc-0305-497e-94a7-3c0080ca7b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:38.096418 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1362/2752] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:38.106360 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:38.107788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:38.172171 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378efc8e-432a-444a-957a-c3e532298fe7 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:38.176033 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3991b89-4b91-42f1-b06b-ef029f83e2cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:38.176642 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991b89-4b91-42f1-b06b-ef029f83e2cd service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:38.176992 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991b89-4b91-42f1-b06b-ef029f83e2cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:38.177267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991b89-4b91-42f1-b06b-ef029f83e2cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:38.186903 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3991b89-4b91-42f1-b06b-ef029f83e2cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:38.188646 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3991b89-4b91-42f1-b06b-ef029f83e2cd service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:38.198699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1391/2753] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:38.597113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c43bfc-0305-497e-94a7-3c0080ca7b71 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:38.598770 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-07c43bfc-0305-497e-94a7-3c0080ca7b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:38.613611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c43bfc-0305-497e-94a7-3c0080ca7b71 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:38.618438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:38.619069 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:38.619351 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:38.619623 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] PATH_INFO: `/v3/projects/d7271f1c9468434e8e6bcb51b6aad1e7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:38.620774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7271f1c9468434e8e6bcb51b6aad1e7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:38.622703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:38.728759 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7271f1c9468434e8e6bcb51b6aad1e7', 'request_id': 'req-83630e0f-aca9-4f24-91c9-78450eb9076f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:38.735562 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7271f1c9468434e8e6bcb51b6aad1e7', 'request_id': 'req-83630e0f-aca9-4f24-91c9-78450eb9076f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:38.742709 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7271f1c9468434e8e6bcb51b6aad1e7', 'request_id': 'req-83630e0f-aca9-4f24-91c9-78450eb9076f'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:38.798628 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] The token cache is being invalidate because project d7271f1c9468434e8e6bcb51b6aad1e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:38.799228 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:38.800228 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:38.802447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1363/2754] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:38 2022] DELETE /identity/v3/projects/d7271f1c9468434e8e6bcb51b6aad1e7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:38.807134 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3991b89-4b91-42f1-b06b-ef029f83e2cd service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:38.821128 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3991b89-4b91-42f1-b06b-ef029f83e2cd service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:38.883930 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62643e2c-2292-4289-82cf-9370bcd85e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:38.884554 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62643e2c-2292-4289-82cf-9370bcd85e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:38.884834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62643e2c-2292-4289-82cf-9370bcd85e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:38.885106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62643e2c-2292-4289-82cf-9370bcd85e82 admin admin] PATH_INFO: `/v3/users/442c531ed5eb425390402393a2a6fc37` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:38.886254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62643e2c-2292-4289-82cf-9370bcd85e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=442c531ed5eb425390402393a2a6fc37)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:38.888168 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62643e2c-2292-4289-82cf-9370bcd85e82 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:39.045425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-62643e2c-2292-4289-82cf-9370bcd85e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '442c531ed5eb425390402393a2a6fc37'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:39.115358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-62643e2c-2292-4289-82cf-9370bcd85e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '442c531ed5eb425390402393a2a6fc37'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:39.116043 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-62643e2c-2292-4289-82cf-9370bcd85e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '442c531ed5eb425390402393a2a6fc37'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:39.123680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-62643e2c-2292-4289-82cf-9370bcd85e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '442c531ed5eb425390402393a2a6fc37'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:39.133578 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:39.140901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-62643e2c-2292-4289-82cf-9370bcd85e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '442c531ed5eb425390402393a2a6fc37'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:39.153299 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1392/2755] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:38 2022] DELETE /identity/v3/users/442c531ed5eb425390402393a2a6fc37 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:39.585359 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:39.587019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:39.602204 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83630e0f-aca9-4f24-91c9-78450eb9076f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:39.648827 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:39.661123 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:39.661792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:39.662073 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:39.662346 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7 admin admin] PATH_INFO: `/v3/projects/abeb4eee85484282963aba4615f6c291` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:39.667992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abeb4eee85484282963aba4615f6c291)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:39.667992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:39.779129 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abeb4eee85484282963aba4615f6c291', 'request_id': 'req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:39.785858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abeb4eee85484282963aba4615f6c291', 'request_id': 'req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:39.792337 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abeb4eee85484282963aba4615f6c291', 'request_id': 'req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:39.886553 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7 admin admin] The token cache is being invalidate because project abeb4eee85484282963aba4615f6c291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:39.886937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:39.887678 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-4ea10a1b-0c54-4b0a-89bb-690efe2046d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:39.889296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1364/2756] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:39 2022] DELETE /identity/v3/projects/abeb4eee85484282963aba4615f6c291 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:39.895379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e4318-5eb0-4092-b749-6be8ce7febac None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:39.895973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e4318-5eb0-4092-b749-6be8ce7febac None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:39.895973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e4318-5eb0-4092-b749-6be8ce7febac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:39.917436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-e71e4318-5eb0-4092-b749-6be8ce7febac None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:39.919739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e71e4318-5eb0-4092-b749-6be8ce7febac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:39.997500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1393/2757] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:40.008667 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf98954-188c-4a87-92b8-dc7ec72ccffc None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:40.010664 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf98954-188c-4a87-92b8-dc7ec72ccffc None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:40.011529 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf98954-188c-4a87-92b8-dc7ec72ccffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:40.045965 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-6cf98954-188c-4a87-92b8-dc7ec72ccffc None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:40.047477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-6cf98954-188c-4a87-92b8-dc7ec72ccffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:40.056769 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1365/2758] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:40.063513 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e4318-5eb0-4092-b749-6be8ce7febac None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:40.078850 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e4318-5eb0-4092-b749-6be8ce7febac None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:40.084183 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2ce20-b441-4004-a40b-4d30adcb53a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:40.084801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2ce20-b441-4004-a40b-4d30adcb53a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:40.085076 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2ce20-b441-4004-a40b-4d30adcb53a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:40.085396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2ce20-b441-4004-a40b-4d30adcb53a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:40.086664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2ce20-b441-4004-a40b-4d30adcb53a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:57:40.087014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2ce20-b441-4004-a40b-4d30adcb53a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:40.088622 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2ce20-b441-4004-a40b-4d30adcb53a3 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:40.112023 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1394/2759] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:57:40.112257 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf98954-188c-4a87-92b8-dc7ec72ccffc None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:40.117592 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf98954-188c-4a87-92b8-dc7ec72ccffc None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:40.123096 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910654a-fbbe-43c0-a53f-5921b11a1af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:40.123699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910654a-fbbe-43c0-a53f-5921b11a1af8 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:40.123980 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910654a-fbbe-43c0-a53f-5921b11a1af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:40.124253 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910654a-fbbe-43c0-a53f-5921b11a1af8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:40.125048 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910654a-fbbe-43c0-a53f-5921b11a1af8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:40.127047 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910654a-fbbe-43c0-a53f-5921b11a1af8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:40.238105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1366/2760] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:57:40 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:57:40.242572 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2ce20-b441-4004-a40b-4d30adcb53a3 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:40.256322 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2ce20-b441-4004-a40b-4d30adcb53a3 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:40.335047 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471479ea-65d1-46d1-bbf6-8177485f967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:40.335569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471479ea-65d1-46d1-bbf6-8177485f967e admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:40.335871 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471479ea-65d1-46d1-bbf6-8177485f967e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:40.336141 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471479ea-65d1-46d1-bbf6-8177485f967e admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:40.336877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471479ea-65d1-46d1-bbf6-8177485f967e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:40.338795 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471479ea-65d1-46d1-bbf6-8177485f967e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:40.424491 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1395/2761] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:57:40 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:40.428629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910654a-fbbe-43c0-a53f-5921b11a1af8 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:40.440093 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910654a-fbbe-43c0-a53f-5921b11a1af8 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:40.444918 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33f938-15b9-4596-9950-762e50e5da1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:40.446240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33f938-15b9-4596-9950-762e50e5da1e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:40.452411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33f938-15b9-4596-9950-762e50e5da1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:40.452411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33f938-15b9-4596-9950-762e50e5da1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:40.452411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33f938-15b9-4596-9950-762e50e5da1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:40.452411 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33f938-15b9-4596-9950-762e50e5da1e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:40.463147 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1367/2762] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:40.471229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471479ea-65d1-46d1-bbf6-8177485f967e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:40.481960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471479ea-65d1-46d1-bbf6-8177485f967e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:40.486839 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a8d9f-12a0-4ab2-a5d2-b7333d41bb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:40.487446 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8d9f-12a0-4ab2-a5d2-b7333d41bb7b admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:40.487729 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8d9f-12a0-4ab2-a5d2-b7333d41bb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:40.488005 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8d9f-12a0-4ab2-a5d2-b7333d41bb7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:40.488650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a8d9f-12a0-4ab2-a5d2-b7333d41bb7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:40.490319 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a8d9f-12a0-4ab2-a5d2-b7333d41bb7b admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:40.501623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1396/2763] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:40.505652 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33f938-15b9-4596-9950-762e50e5da1e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:40.515671 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33f938-15b9-4596-9950-762e50e5da1e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:40.520201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf509a-bfdc-4585-bafc-a87f57a80a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:40.520841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf509a-bfdc-4585-bafc-a87f57a80a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:40.521120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf509a-bfdc-4585-bafc-a87f57a80a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:40.521394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf509a-bfdc-4585-bafc-a87f57a80a03 admin admin] PATH_INFO: `/v3/projects/2354ec07535c4b1f8c7908ad243038c8/users/ef6c2fb3147440f59954c5a52fa4c261/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:40.535969 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf509a-bfdc-4585-bafc-a87f57a80a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2354ec07535c4b1f8c7908ad243038c8, user_id=ef6c2fb3147440f59954c5a52fa4c261, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:40.538690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf509a-bfdc-4585-bafc-a87f57a80a03 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:40.595482 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1368/2764] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:40 2022] PUT /identity/v3/projects/2354ec07535c4b1f8c7908ad243038c8/users/ef6c2fb3147440f59954c5a52fa4c261/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:40.607729 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81f93f-b4d7-44e2-af70-41ada2a479df None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:40.608022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81f93f-b4d7-44e2-af70-41ada2a479df None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:40.608298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81f93f-b4d7-44e2-af70-41ada2a479df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:40.637027 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5e81f93f-b4d7-44e2-af70-41ada2a479df None None] MFA Rules not processed for user `ef6c2fb3147440f59954c5a52fa4c261`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:40.638198 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5e81f93f-b4d7-44e2-af70-41ada2a479df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:40.690894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:40.714669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1397/2765] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:40.719223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf509a-bfdc-4585-bafc-a87f57a80a03 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:40.783567 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf509a-bfdc-4585-bafc-a87f57a80a03 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:40.788360 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200db97-56a8-43a5-aa8b-077a783f6ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:40.789511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200db97-56a8-43a5-aa8b-077a783f6ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:40.789821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200db97-56a8-43a5-aa8b-077a783f6ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:40.790097 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200db97-56a8-43a5-aa8b-077a783f6ac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:40.790743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200db97-56a8-43a5-aa8b-077a783f6ac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:40.792637 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200db97-56a8-43a5-aa8b-077a783f6ac6 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:40.881444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:40.888514 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1369/2766] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:57:40 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:40.892618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81f93f-b4d7-44e2-af70-41ada2a479df None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:40.904093 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81f93f-b4d7-44e2-af70-41ada2a479df None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:40.960680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09346d-cc97-446e-a212-197cdb2ab2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:40.962034 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09346d-cc97-446e-a212-197cdb2ab2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:40.962316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09346d-cc97-446e-a212-197cdb2ab2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:40.962591 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09346d-cc97-446e-a212-197cdb2ab2a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:40.963221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09346d-cc97-446e-a212-197cdb2ab2a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:40.965147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09346d-cc97-446e-a212-197cdb2ab2a2 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:41.041546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1398/2767] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:57:40 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:57:41.045289 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200db97-56a8-43a5-aa8b-077a783f6ac6 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:41.056756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200db97-56a8-43a5-aa8b-077a783f6ac6 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:41.062037 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca154c6-3486-4757-8018-ad5f23264298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:41.063131 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca154c6-3486-4757-8018-ad5f23264298 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:41.063473 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca154c6-3486-4757-8018-ad5f23264298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:41.063750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca154c6-3486-4757-8018-ad5f23264298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:41.064398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca154c6-3486-4757-8018-ad5f23264298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:41.066107 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca154c6-3486-4757-8018-ad5f23264298 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:41.078462 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1370/2768] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:41.082332 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09346d-cc97-446e-a212-197cdb2ab2a2 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:41.093331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09346d-cc97-446e-a212-197cdb2ab2a2 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:41.098752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381379a-da01-4958-b4a3-0b2ad2951128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:41.099420 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381379a-da01-4958-b4a3-0b2ad2951128 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:41.099883 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381379a-da01-4958-b4a3-0b2ad2951128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:41.100170 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381379a-da01-4958-b4a3-0b2ad2951128 admin admin] PATH_INFO: `/v3/projects/08c2e3542ed6461b9251cedacd1d9f9d/users/f59dbdc3ced64af78923eec17e236f31/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:41.119906 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381379a-da01-4958-b4a3-0b2ad2951128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c2e3542ed6461b9251cedacd1d9f9d, user_id=f59dbdc3ced64af78923eec17e236f31, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:41.123071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381379a-da01-4958-b4a3-0b2ad2951128 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:41.172593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1399/2769] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:41 2022] PUT /identity/v3/projects/08c2e3542ed6461b9251cedacd1d9f9d/users/f59dbdc3ced64af78923eec17e236f31/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:41.176444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca154c6-3486-4757-8018-ad5f23264298 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:41.186604 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca154c6-3486-4757-8018-ad5f23264298 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:41.252354 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fbb3c-ae79-4a9a-9dc5-fa84c44cd334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:41.253370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fbb3c-ae79-4a9a-9dc5-fa84c44cd334 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:41.253884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fbb3c-ae79-4a9a-9dc5-fa84c44cd334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:41.254351 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fbb3c-ae79-4a9a-9dc5-fa84c44cd334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:41.255909 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fbb3c-ae79-4a9a-9dc5-fa84c44cd334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:41.259386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fbb3c-ae79-4a9a-9dc5-fa84c44cd334 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:41.273425 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1371/2770] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:57:41.285213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381379a-da01-4958-b4a3-0b2ad2951128 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:41.295755 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381379a-da01-4958-b4a3-0b2ad2951128 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:41.300725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541735d8-04dc-4d20-9005-0403ffd6b51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:41.301379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541735d8-04dc-4d20-9005-0403ffd6b51d admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:41.301701 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541735d8-04dc-4d20-9005-0403ffd6b51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:41.301979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541735d8-04dc-4d20-9005-0403ffd6b51d admin admin] PATH_INFO: `/v3/projects/08c2e3542ed6461b9251cedacd1d9f9d/users/f59dbdc3ced64af78923eec17e236f31/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:41.304901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541735d8-04dc-4d20-9005-0403ffd6b51d admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c2e3542ed6461b9251cedacd1d9f9d, user_id=f59dbdc3ced64af78923eec17e236f31, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:41.308051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541735d8-04dc-4d20-9005-0403ffd6b51d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:41.322815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1400/2771] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:41 2022] PUT /identity/v3/projects/08c2e3542ed6461b9251cedacd1d9f9d/users/f59dbdc3ced64af78923eec17e236f31/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:41.395883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4da66-4b63-4ae1-9708-adb728bc2263 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:41.396521 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4da66-4b63-4ae1-9708-adb728bc2263 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:41.396704 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4da66-4b63-4ae1-9708-adb728bc2263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:41.427033 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-94c4da66-4b63-4ae1-9708-adb728bc2263 None None] MFA Rules not processed for user `f59dbdc3ced64af78923eec17e236f31`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:41.428243 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-94c4da66-4b63-4ae1-9708-adb728bc2263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:41.502778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1372/2772] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:57:41.514480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541735d8-04dc-4d20-9005-0403ffd6b51d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:41.517549 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-541735d8-04dc-4d20-9005-0403ffd6b51d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:41.581425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541735d8-04dc-4d20-9005-0403ffd6b51d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:41.585365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07091864-371d-42ef-b2f9-8da3131020ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:41.586632 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07091864-371d-42ef-b2f9-8da3131020ba service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:41.590352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07091864-371d-42ef-b2f9-8da3131020ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:41.590352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07091864-371d-42ef-b2f9-8da3131020ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:41.645392 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:41.649199 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07091864-371d-42ef-b2f9-8da3131020ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:41.650732 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07091864-371d-42ef-b2f9-8da3131020ba service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:41.660122 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1401/2773] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:57:46.934291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c4da66-4b63-4ae1-9708-adb728bc2263 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:46.955508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62febbd1-b46d-439f-9f51-fb8939079ce8 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:46.956078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62febbd1-b46d-439f-9f51-fb8939079ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:46.956078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62febbd1-b46d-439f-9f51-fb8939079ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:46.975552 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-62febbd1-b46d-439f-9f51-fb8939079ce8 None None] MFA Rules not processed for user `aa415c4e0e9c4928b51152fa0a428d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:46.976810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-62febbd1-b46d-439f-9f51-fb8939079ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:47.007908 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c4da66-4b63-4ae1-9708-adb728bc2263 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:47.013393 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:47.014054 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:47.014332 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:47.014602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] PATH_INFO: `/v3/users/ef6c2fb3147440f59954c5a52fa4c261` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:47.015937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6c2fb3147440f59954c5a52fa4c261)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:47.017943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:47.031185 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:47.035508 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1402/2774] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:57:47.258713 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6c2fb3147440f59954c5a52fa4c261'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:47.350590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6c2fb3147440f59954c5a52fa4c261'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:47.351670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6c2fb3147440f59954c5a52fa4c261'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:47.363789 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6c2fb3147440f59954c5a52fa4c261'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:47.383999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6c2fb3147440f59954c5a52fa4c261'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:47.402685 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1373/2775] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:46 2022] DELETE /identity/v3/users/ef6c2fb3147440f59954c5a52fa4c261 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:47.404652 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410490b-0527-4a27-bfe0-cb7127ee65e6 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:47.405092 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410490b-0527-4a27-bfe0-cb7127ee65e6 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:47.405405 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410490b-0527-4a27-bfe0-cb7127ee65e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:47.429229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-0410490b-0527-4a27-bfe0-cb7127ee65e6 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:47.429229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0410490b-0527-4a27-bfe0-cb7127ee65e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:47.520748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1403/2776] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:47.533599 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:47.535559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:47.607471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc389a86-5e63-4468-8cdf-46d5caf2a045 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:47.611620 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b1cf7-e817-4128-a5cc-b8496feba9a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:47.612234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b1cf7-e817-4128-a5cc-b8496feba9a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:47.612511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b1cf7-e817-4128-a5cc-b8496feba9a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:47.612955 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b1cf7-e817-4128-a5cc-b8496feba9a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:47.623589 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b1cf7-e817-4128-a5cc-b8496feba9a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:47.625043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b1cf7-e817-4128-a5cc-b8496feba9a1 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:47.635720 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1374/2777] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:57:48.238437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410490b-0527-4a27-bfe0-cb7127ee65e6 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:48.239980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-0410490b-0527-4a27-bfe0-cb7127ee65e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:48.260132 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410490b-0527-4a27-bfe0-cb7127ee65e6 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:48.260352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:48.261000 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:48.261278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:48.261564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] PATH_INFO: `/v3/projects/2354ec07535c4b1f8c7908ad243038c8` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:48.263437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2354ec07535c4b1f8c7908ad243038c8)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:48.265885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:48.462737 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2354ec07535c4b1f8c7908ad243038c8', 'request_id': 'req-c66c67ef-9f1a-47dc-96f5-16345fbb4143'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:48.475071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2354ec07535c4b1f8c7908ad243038c8', 'request_id': 'req-c66c67ef-9f1a-47dc-96f5-16345fbb4143'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:48.484560 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2354ec07535c4b1f8c7908ad243038c8', 'request_id': 'req-c66c67ef-9f1a-47dc-96f5-16345fbb4143'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:48.595588 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] The token cache is being invalidate because project 2354ec07535c4b1f8c7908ad243038c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:48.596137 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:48.597371 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:48.600618 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1404/2778] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:48 2022] DELETE /identity/v3/projects/2354ec07535c4b1f8c7908ad243038c8 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:48.605194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b1cf7-e817-4128-a5cc-b8496feba9a1 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:48.615008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b1cf7-e817-4128-a5cc-b8496feba9a1 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:48.672622 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2619dfd-cd91-48bf-b49f-d7a504155339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:48.673256 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2619dfd-cd91-48bf-b49f-d7a504155339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:48.673588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2619dfd-cd91-48bf-b49f-d7a504155339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:48.673986 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2619dfd-cd91-48bf-b49f-d7a504155339 admin admin] PATH_INFO: `/v3/users/f59dbdc3ced64af78923eec17e236f31` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:48.675148 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2619dfd-cd91-48bf-b49f-d7a504155339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f59dbdc3ced64af78923eec17e236f31)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:48.677084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2619dfd-cd91-48bf-b49f-d7a504155339 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:48.922754 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:48.936684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b2619dfd-cd91-48bf-b49f-d7a504155339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f59dbdc3ced64af78923eec17e236f31'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:49.041052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b2619dfd-cd91-48bf-b49f-d7a504155339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f59dbdc3ced64af78923eec17e236f31'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:49.041732 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b2619dfd-cd91-48bf-b49f-d7a504155339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f59dbdc3ced64af78923eec17e236f31'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:49.049700 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b2619dfd-cd91-48bf-b49f-d7a504155339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f59dbdc3ced64af78923eec17e236f31'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:49.062187 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-b2619dfd-cd91-48bf-b49f-d7a504155339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f59dbdc3ced64af78923eec17e236f31'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:49.068962 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1375/2779] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:57:48 2022] DELETE /identity/v3/users/f59dbdc3ced64af78923eec17e236f31 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:57:49.509355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:49.510914 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:49.511682 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a06113-b83e-4187-8544-eb4dad5ec615 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:49.511976 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a06113-b83e-4187-8544-eb4dad5ec615 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:49.512244 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a06113-b83e-4187-8544-eb4dad5ec615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:49.530452 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c67ef-9f1a-47dc-96f5-16345fbb4143 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:49.531983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-d2a06113-b83e-4187-8544-eb4dad5ec615 None None] MFA Rules not processed for user `debdf689e44c48e1bca1c7ad37f8335f`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:49.533288 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-d2a06113-b83e-4187-8544-eb4dad5ec615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:49.555936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:49.593581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1376/2780] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:49.594562 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcfa06-99ee-481c-8556-73c174f02dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:49.595213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcfa06-99ee-481c-8556-73c174f02dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:49.595527 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcfa06-99ee-481c-8556-73c174f02dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:49.595811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcfa06-99ee-481c-8556-73c174f02dc8 admin admin] PATH_INFO: `/v3/projects/08c2e3542ed6461b9251cedacd1d9f9d` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:49.596984 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcfa06-99ee-481c-8556-73c174f02dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c2e3542ed6461b9251cedacd1d9f9d)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:49.599131 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcfa06-99ee-481c-8556-73c174f02dc8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:49.702065 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-16bcfa06-99ee-481c-8556-73c174f02dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c2e3542ed6461b9251cedacd1d9f9d', 'request_id': 'req-16bcfa06-99ee-481c-8556-73c174f02dc8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:49.708052 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-16bcfa06-99ee-481c-8556-73c174f02dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c2e3542ed6461b9251cedacd1d9f9d', 'request_id': 'req-16bcfa06-99ee-481c-8556-73c174f02dc8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:49.715114 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-16bcfa06-99ee-481c-8556-73c174f02dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c2e3542ed6461b9251cedacd1d9f9d', 'request_id': 'req-16bcfa06-99ee-481c-8556-73c174f02dc8'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:49.823793 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-16bcfa06-99ee-481c-8556-73c174f02dc8 admin admin] The token cache is being invalidate because project 08c2e3542ed6461b9251cedacd1d9f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:57:49.824071 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-16bcfa06-99ee-481c-8556-73c174f02dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:49.824707 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-16bcfa06-99ee-481c-8556-73c174f02dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:57:49.826242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1405/2781] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:57:49 2022] DELETE /identity/v3/projects/08c2e3542ed6461b9251cedacd1d9f9d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:57:49.831460 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142345d-98b7-45af-b4d5-9eadd2fe9f52 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:49.831998 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142345d-98b7-45af-b4d5-9eadd2fe9f52 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:49.832128 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142345d-98b7-45af-b4d5-9eadd2fe9f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:49.853507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-2142345d-98b7-45af-b4d5-9eadd2fe9f52 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:49.854750 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-2142345d-98b7-45af-b4d5-9eadd2fe9f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:49.945573 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1377/2782] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:57:49.952853 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154e479-298a-4129-8f7b-e34c8b1d921c None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:49.953140 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154e479-298a-4129-8f7b-e34c8b1d921c None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:49.953421 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154e479-298a-4129-8f7b-e34c8b1d921c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:49.983229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-2154e479-298a-4129-8f7b-e34c8b1d921c None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:49.984703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-2154e479-298a-4129-8f7b-e34c8b1d921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:49.995313 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1406/2783] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:49.999542 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142345d-98b7-45af-b4d5-9eadd2fe9f52 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:50.010470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142345d-98b7-45af-b4d5-9eadd2fe9f52 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:50.015896 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16d7bc-82ca-4e44-9a82-0a1fd8f53095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:50.016515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16d7bc-82ca-4e44-9a82-0a1fd8f53095 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:50.016928 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16d7bc-82ca-4e44-9a82-0a1fd8f53095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:50.017212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16d7bc-82ca-4e44-9a82-0a1fd8f53095 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:50.017907 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16d7bc-82ca-4e44-9a82-0a1fd8f53095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:57:50.018212 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16d7bc-82ca-4e44-9a82-0a1fd8f53095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:50.019845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16d7bc-82ca-4e44-9a82-0a1fd8f53095 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:50.035724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1378/2784] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:57:50.040691 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154e479-298a-4129-8f7b-e34c8b1d921c None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:50.051998 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154e479-298a-4129-8f7b-e34c8b1d921c None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:50.058926 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ea6ad-d5e3-42ba-99fb-b7ab5d8e85ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:50.059546 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ea6ad-d5e3-42ba-99fb-b7ab5d8e85ac admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:50.059983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ea6ad-d5e3-42ba-99fb-b7ab5d8e85ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:50.060259 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ea6ad-d5e3-42ba-99fb-b7ab5d8e85ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:50.060949 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ea6ad-d5e3-42ba-99fb-b7ab5d8e85ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:50.063814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ea6ad-d5e3-42ba-99fb-b7ab5d8e85ac admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:50.189930 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1407/2785] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:57:50 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:50.194090 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16d7bc-82ca-4e44-9a82-0a1fd8f53095 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:50.222735 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16d7bc-82ca-4e44-9a82-0a1fd8f53095 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:50.272970 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:50.290977 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1ca12-5448-4fac-a4e7-f88a0a68841f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:50.291689 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1ca12-5448-4fac-a4e7-f88a0a68841f admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:50.292039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1ca12-5448-4fac-a4e7-f88a0a68841f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:50.292387 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1ca12-5448-4fac-a4e7-f88a0a68841f admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:50.293091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1ca12-5448-4fac-a4e7-f88a0a68841f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:50.295064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1ca12-5448-4fac-a4e7-f88a0a68841f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:50.420805 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1379/2786] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:57:50 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:57:50.426889 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ea6ad-d5e3-42ba-99fb-b7ab5d8e85ac admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:50.448712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ea6ad-d5e3-42ba-99fb-b7ab5d8e85ac admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:50.454784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f78dc-be13-4374-868f-33b8a58f83d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:50.455775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f78dc-be13-4374-868f-33b8a58f83d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:50.456055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f78dc-be13-4374-868f-33b8a58f83d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:50.456326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f78dc-be13-4374-868f-33b8a58f83d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:50.456975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f78dc-be13-4374-868f-33b8a58f83d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:50.458750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f78dc-be13-4374-868f-33b8a58f83d8 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:50.472486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1408/2787] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:50.474762 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1ca12-5448-4fac-a4e7-f88a0a68841f admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:50.487623 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1ca12-5448-4fac-a4e7-f88a0a68841f admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:50.494882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecb4f9-7970-4968-95fc-7ba00c1d2a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:50.494882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecb4f9-7970-4968-95fc-7ba00c1d2a9f admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:50.494882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecb4f9-7970-4968-95fc-7ba00c1d2a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:50.494882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecb4f9-7970-4968-95fc-7ba00c1d2a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:50.495257 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecb4f9-7970-4968-95fc-7ba00c1d2a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:50.496901 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecb4f9-7970-4968-95fc-7ba00c1d2a9f admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:50.509439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1380/2788] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:57:50.513430 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f78dc-be13-4374-868f-33b8a58f83d8 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:50.527581 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f78dc-be13-4374-868f-33b8a58f83d8 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:50.533870 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca71a8-1122-4a4e-a550-d92b9035324d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:50.534535 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca71a8-1122-4a4e-a550-d92b9035324d admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:50.534822 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca71a8-1122-4a4e-a550-d92b9035324d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:50.535097 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca71a8-1122-4a4e-a550-d92b9035324d admin admin] PATH_INFO: `/v3/projects/e1d6bc7e07c74f4db8247fe2ffff60d9/users/9de00b3f072d4fe9bbb8f3d1a2409bc3/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:50.553105 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca71a8-1122-4a4e-a550-d92b9035324d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d6bc7e07c74f4db8247fe2ffff60d9, user_id=9de00b3f072d4fe9bbb8f3d1a2409bc3, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:50.553379 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca71a8-1122-4a4e-a550-d92b9035324d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:50.676478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1409/2789] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:57:50 2022] PUT /identity/v3/projects/e1d6bc7e07c74f4db8247fe2ffff60d9/users/9de00b3f072d4fe9bbb8f3d1a2409bc3/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:57:50.689162 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00fc2-70dd-4e9b-912b-47ac39f0175d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:50.689451 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00fc2-70dd-4e9b-912b-47ac39f0175d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:50.689769 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00fc2-70dd-4e9b-912b-47ac39f0175d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:50.720396 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-88b00fc2-70dd-4e9b-912b-47ac39f0175d None None] MFA Rules not processed for user `9de00b3f072d4fe9bbb8f3d1a2409bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:57:50.720396 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-88b00fc2-70dd-4e9b-912b-47ac39f0175d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:50.795051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1381/2790] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:57:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:57:50.807994 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca71a8-1122-4a4e-a550-d92b9035324d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:57:50.809765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-b9ca71a8-1122-4a4e-a550-d92b9035324d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:57:50.868916 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:57:50.873316 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca71a8-1122-4a4e-a550-d92b9035324d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:57:50.877213 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f62ada-7c14-4f70-964a-e7998d2f5b05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:57:50.877887 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f62ada-7c14-4f70-964a-e7998d2f5b05 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:57:50.878235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f62ada-7c14-4f70-964a-e7998d2f5b05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:57:50.878493 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f62ada-7c14-4f70-964a-e7998d2f5b05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:57:50.888730 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f62ada-7c14-4f70-964a-e7998d2f5b05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:57:50.900054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f62ada-7c14-4f70-964a-e7998d2f5b05 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:57:50.900054 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1410/2791] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:57:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:58:00.792130 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:58:01.120993 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:58:15.771702 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00fc2-70dd-4e9b-912b-47ac39f0175d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:15.836296 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00fc2-70dd-4e9b-912b-47ac39f0175d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:15.841615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:15.842245 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:15.842525 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:15.842796 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] PATH_INFO: `/v3/users/9de00b3f072d4fe9bbb8f3d1a2409bc3` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:15.843912 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9de00b3f072d4fe9bbb8f3d1a2409bc3)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:15.845905 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:16.049748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9de00b3f072d4fe9bbb8f3d1a2409bc3'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:16.120738 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9de00b3f072d4fe9bbb8f3d1a2409bc3'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:16.121894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9de00b3f072d4fe9bbb8f3d1a2409bc3'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:16.130364 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9de00b3f072d4fe9bbb8f3d1a2409bc3'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:16.144002 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9de00b3f072d4fe9bbb8f3d1a2409bc3'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:16.152494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1382/2792] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:58:15 2022] DELETE /identity/v3/users/9de00b3f072d4fe9bbb8f3d1a2409bc3 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:58:16.158120 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074af735-2e6b-402c-8665-b0edd84c5ac9 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:16.160765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074af735-2e6b-402c-8665-b0edd84c5ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:16.160765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074af735-2e6b-402c-8665-b0edd84c5ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:16.179061 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-074af735-2e6b-402c-8665-b0edd84c5ac9 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:58:16.180771 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-074af735-2e6b-402c-8665-b0edd84c5ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:16.244366 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1411/2793] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:58:16.261893 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:16.263821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:16.334876 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3325b-a411-495c-b06f-a47694991cc7 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:16.338757 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d4593-b2a0-4af3-b33e-182f33a38609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:16.339391 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d4593-b2a0-4af3-b33e-182f33a38609 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:16.339669 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d4593-b2a0-4af3-b33e-182f33a38609 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:16.339937 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d4593-b2a0-4af3-b33e-182f33a38609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:16.349598 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d4593-b2a0-4af3-b33e-182f33a38609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:16.351022 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d4593-b2a0-4af3-b33e-182f33a38609 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:16.360670 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1383/2794] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:58:16.871389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074af735-2e6b-402c-8665-b0edd84c5ac9 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:16.872853 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-074af735-2e6b-402c-8665-b0edd84c5ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:16.885612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074af735-2e6b-402c-8665-b0edd84c5ac9 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:16.891113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:16.891740 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:16.892016 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:16.892592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] PATH_INFO: `/v3/projects/e1d6bc7e07c74f4db8247fe2ffff60d9` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:16.893603 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1d6bc7e07c74f4db8247fe2ffff60d9)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:16.895534 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:17.001335 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1d6bc7e07c74f4db8247fe2ffff60d9', 'request_id': 'req-16435c3d-bf52-4947-8978-071fc5a3ae6c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:17.006756 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1d6bc7e07c74f4db8247fe2ffff60d9', 'request_id': 'req-16435c3d-bf52-4947-8978-071fc5a3ae6c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:17.012804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1d6bc7e07c74f4db8247fe2ffff60d9', 'request_id': 'req-16435c3d-bf52-4947-8978-071fc5a3ae6c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:17.104480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] The token cache is being invalidate because project e1d6bc7e07c74f4db8247fe2ffff60d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:58:17.104857 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:17.118315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:17.118315 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1412/2795] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:58:16 2022] DELETE /identity/v3/projects/e1d6bc7e07c74f4db8247fe2ffff60d9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:58:25.670292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea206a5a-6897-44bb-a929-0f764929254e None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:25.670783 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea206a5a-6897-44bb-a929-0f764929254e None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:25.670884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea206a5a-6897-44bb-a929-0f764929254e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:25.701234 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ea206a5a-6897-44bb-a929-0f764929254e None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:58:25.705673 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ea206a5a-6897-44bb-a929-0f764929254e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:25.711014 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:58:25.766721 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1384/2796] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:58:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:58:25.807617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:25.809804 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:25.885564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16435c3d-bf52-4947-8978-071fc5a3ae6c admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:25.890760 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd3252-5fe7-4b7f-9ee7-927ae848f67e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:25.891367 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd3252-5fe7-4b7f-9ee7-927ae848f67e service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:25.891638 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd3252-5fe7-4b7f-9ee7-927ae848f67e service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:25.891911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd3252-5fe7-4b7f-9ee7-927ae848f67e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:25.899961 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd3252-5fe7-4b7f-9ee7-927ae848f67e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:25.901571 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd3252-5fe7-4b7f-9ee7-927ae848f67e service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:25.910212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1413/2797] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:58:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:58:35.857726 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=22 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:58:35.908975 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:58:43.190740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea5db4-6346-490f-b225-b738561a55ec None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:43.191445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea5db4-6346-490f-b225-b738561a55ec None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:43.191445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea5db4-6346-490f-b225-b738561a55ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:43.211013 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-76ea5db4-6346-490f-b225-b738561a55ec None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:58:43.212194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-76ea5db4-6346-490f-b225-b738561a55ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:43.219817 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1385/2798] 10.222.0.57 () {58 vars in 967 bytes} [Tue Aug 9 18:58:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:58:43.232197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd3252-5fe7-4b7f-9ee7-927ae848f67e service swift] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:43.243911 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd3252-5fe7-4b7f-9ee7-927ae848f67e service swift] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:43.249325 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f35255-5c04-418e-807a-f61c7a7d4e1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:43.249833 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f35255-5c04-418e-807a-f61c7a7d4e1b service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:43.250127 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f35255-5c04-418e-807a-f61c7a7d4e1b service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:43.250399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f35255-5c04-418e-807a-f61c7a7d4e1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:43.259397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f35255-5c04-418e-807a-f61c7a7d4e1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:43.261011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f35255-5c04-418e-807a-f61c7a7d4e1b service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:43.270858 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1414/2799] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:58:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:58:46.973493 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea5db4-6346-490f-b225-b738561a55ec None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:46.975043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-76ea5db4-6346-490f-b225-b738561a55ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:47.033989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea5db4-6346-490f-b225-b738561a55ec None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:47.037515 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8de25d-ea8f-444d-9f20-fb5f70c377e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:47.038142 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8de25d-ea8f-444d-9f20-fb5f70c377e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:47.038424 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8de25d-ea8f-444d-9f20-fb5f70c377e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:47.038697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8de25d-ea8f-444d-9f20-fb5f70c377e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:47.040480 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4f8de25d-ea8f-444d-9f20-fb5f70c377e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:47.066827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f35255-5c04-418e-807a-f61c7a7d4e1b service swift] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:47.068443 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c3f35255-5c04-418e-807a-f61c7a7d4e1b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:47.132055 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8de25d-ea8f-444d-9f20-fb5f70c377e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:47.133552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8de25d-ea8f-444d-9f20-fb5f70c377e0 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:47.141885 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f35255-5c04-418e-807a-f61c7a7d4e1b service swift] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:47.146784 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71081a73-7f5a-45c2-bb34-536f45a807e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:47.147406 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71081a73-7f5a-45c2-bb34-536f45a807e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:47.147680 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71081a73-7f5a-45c2-bb34-536f45a807e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:47.147948 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71081a73-7f5a-45c2-bb34-536f45a807e4 admin admin] PATH_INFO: `/v3/users/debdf689e44c48e1bca1c7ad37f8335f` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:47.149055 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71081a73-7f5a-45c2-bb34-536f45a807e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=debdf689e44c48e1bca1c7ad37f8335f)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:47.151058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71081a73-7f5a-45c2-bb34-536f45a807e4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:47.155113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1386/2800] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:58:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:58:47.447712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-71081a73-7f5a-45c2-bb34-536f45a807e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'debdf689e44c48e1bca1c7ad37f8335f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:47.589783 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-71081a73-7f5a-45c2-bb34-536f45a807e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'debdf689e44c48e1bca1c7ad37f8335f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:47.590658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-71081a73-7f5a-45c2-bb34-536f45a807e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'debdf689e44c48e1bca1c7ad37f8335f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:47.598229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-71081a73-7f5a-45c2-bb34-536f45a807e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'debdf689e44c48e1bca1c7ad37f8335f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:47.611788 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-71081a73-7f5a-45c2-bb34-536f45a807e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'debdf689e44c48e1bca1c7ad37f8335f'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:47.617973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1415/2801] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:58:47 2022] DELETE /identity/v3/users/debdf689e44c48e1bca1c7ad37f8335f => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:58:48.123483 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8de25d-ea8f-444d-9f20-fb5f70c377e0 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:48.136520 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8de25d-ea8f-444d-9f20-fb5f70c377e0 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:48.226445 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39026de4-2921-4b8c-a00c-c514f4629263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:48.227276 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39026de4-2921-4b8c-a00c-c514f4629263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:48.227588 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39026de4-2921-4b8c-a00c-c514f4629263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:48.227866 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39026de4-2921-4b8c-a00c-c514f4629263 admin admin] PATH_INFO: `/v3/projects/8b6338088b03415aa67ca34647ca0700` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:48.229051 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39026de4-2921-4b8c-a00c-c514f4629263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b6338088b03415aa67ca34647ca0700)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:48.231008 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39026de4-2921-4b8c-a00c-c514f4629263 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:48.252311 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=100 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:58:48.361684 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-39026de4-2921-4b8c-a00c-c514f4629263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b6338088b03415aa67ca34647ca0700', 'request_id': 'req-39026de4-2921-4b8c-a00c-c514f4629263'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:48.368019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-39026de4-2921-4b8c-a00c-c514f4629263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b6338088b03415aa67ca34647ca0700', 'request_id': 'req-39026de4-2921-4b8c-a00c-c514f4629263'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:48.373273 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-39026de4-2921-4b8c-a00c-c514f4629263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b6338088b03415aa67ca34647ca0700', 'request_id': 'req-39026de4-2921-4b8c-a00c-c514f4629263'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:48.520585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-39026de4-2921-4b8c-a00c-c514f4629263 admin admin] The token cache is being invalidate because project 8b6338088b03415aa67ca34647ca0700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:58:48.520875 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-39026de4-2921-4b8c-a00c-c514f4629263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:48.526583 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-39026de4-2921-4b8c-a00c-c514f4629263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:48.526583 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1387/2802] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:58:48 2022] DELETE /identity/v3/projects/8b6338088b03415aa67ca34647ca0700 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:58:48.528907 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06413aaa-2877-4bd0-877f-2ad7cdcb9263 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:48.529374 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06413aaa-2877-4bd0-877f-2ad7cdcb9263 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:48.529627 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06413aaa-2877-4bd0-877f-2ad7cdcb9263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:48.549941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-06413aaa-2877-4bd0-877f-2ad7cdcb9263 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:58:48.551162 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-06413aaa-2877-4bd0-877f-2ad7cdcb9263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:48.613698 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1416/2803] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:58:48.621374 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cd98c-09eb-42b2-8e98-f9e43458d3a6 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:48.621707 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cd98c-09eb-42b2-8e98-f9e43458d3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:48.623858 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cd98c-09eb-42b2-8e98-f9e43458d3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:48.642813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-b65cd98c-09eb-42b2-8e98-f9e43458d3a6 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:58:48.644101 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-b65cd98c-09eb-42b2-8e98-f9e43458d3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:48.652744 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1388/2804] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:58:48.661197 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06413aaa-2877-4bd0-877f-2ad7cdcb9263 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:48.667414 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06413aaa-2877-4bd0-877f-2ad7cdcb9263 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:48.673364 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbca48d-18cf-430c-b6df-812d6c7bc9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:48.674935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbca48d-18cf-430c-b6df-812d6c7bc9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:48.675242 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbca48d-18cf-430c-b6df-812d6c7bc9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:48.685152 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbca48d-18cf-430c-b6df-812d6c7bc9e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:48.688291 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbca48d-18cf-430c-b6df-812d6c7bc9e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:58:48.688617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbca48d-18cf-430c-b6df-812d6c7bc9e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:48.690583 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbca48d-18cf-430c-b6df-812d6c7bc9e4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:48.692361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:58:48.707036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1417/2805] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:58:48.709353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cd98c-09eb-42b2-8e98-f9e43458d3a6 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:48.720552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cd98c-09eb-42b2-8e98-f9e43458d3a6 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:48.726435 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8323aee-a0f0-4db7-9ca8-4487ec134c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:48.727132 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8323aee-a0f0-4db7-9ca8-4487ec134c2c admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:48.727489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8323aee-a0f0-4db7-9ca8-4487ec134c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:48.727842 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8323aee-a0f0-4db7-9ca8-4487ec134c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:48.728548 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8323aee-a0f0-4db7-9ca8-4487ec134c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:48.730560 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8323aee-a0f0-4db7-9ca8-4487ec134c2c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:48.854476 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1389/2806] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:58:48 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:58:48.859311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbca48d-18cf-430c-b6df-812d6c7bc9e4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:48.869625 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbca48d-18cf-430c-b6df-812d6c7bc9e4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:48.932290 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c3187-5071-4e29-8376-a492cfe2cfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:48.932901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c3187-5071-4e29-8376-a492cfe2cfef admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:48.933186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c3187-5071-4e29-8376-a492cfe2cfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:48.933463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c3187-5071-4e29-8376-a492cfe2cfef admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:48.934190 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c3187-5071-4e29-8376-a492cfe2cfef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:48.936752 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c3187-5071-4e29-8376-a492cfe2cfef admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:49.070509 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1418/2807] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:58:48 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:58:49.074791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8323aee-a0f0-4db7-9ca8-4487ec134c2c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:49.085847 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8323aee-a0f0-4db7-9ca8-4487ec134c2c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:49.090585 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e18840-51a1-433c-a082-64a5d54a5dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:49.091194 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e18840-51a1-433c-a082-64a5d54a5dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:49.091471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e18840-51a1-433c-a082-64a5d54a5dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:49.091743 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e18840-51a1-433c-a082-64a5d54a5dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:49.092386 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e18840-51a1-433c-a082-64a5d54a5dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:49.094160 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e18840-51a1-433c-a082-64a5d54a5dbc admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:49.106778 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1390/2808] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:58:49.111924 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c3187-5071-4e29-8376-a492cfe2cfef admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:49.122963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c3187-5071-4e29-8376-a492cfe2cfef admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:49.129779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fd667-6426-42b8-b1d4-590296caa71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:49.129779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fd667-6426-42b8-b1d4-590296caa71f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:49.129779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fd667-6426-42b8-b1d4-590296caa71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:49.129779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fd667-6426-42b8-b1d4-590296caa71f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:49.130302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fd667-6426-42b8-b1d4-590296caa71f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:49.131437 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fd667-6426-42b8-b1d4-590296caa71f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:49.143675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1419/2809] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:58:49.147989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e18840-51a1-433c-a082-64a5d54a5dbc admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:49.159023 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e18840-51a1-433c-a082-64a5d54a5dbc admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:49.164294 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c09e80-944f-4fbb-aed7-af2f8ec00a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:49.164953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c09e80-944f-4fbb-aed7-af2f8ec00a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:49.165239 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c09e80-944f-4fbb-aed7-af2f8ec00a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:49.165802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c09e80-944f-4fbb-aed7-af2f8ec00a1b admin admin] PATH_INFO: `/v3/projects/973115e67d7749d785795773d18de181/users/581d7e60bfdc49a9b492d6c89ae9268e/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:49.183724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c09e80-944f-4fbb-aed7-af2f8ec00a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=973115e67d7749d785795773d18de181, user_id=581d7e60bfdc49a9b492d6c89ae9268e, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:49.187298 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c09e80-944f-4fbb-aed7-af2f8ec00a1b admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:49.327611 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1391/2810] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:58:49 2022] PUT /identity/v3/projects/973115e67d7749d785795773d18de181/users/581d7e60bfdc49a9b492d6c89ae9268e/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:58:49.337320 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da2416-3df5-43c4-bd06-1bb824d888bc None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:49.337648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da2416-3df5-43c4-bd06-1bb824d888bc None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:49.337927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da2416-3df5-43c4-bd06-1bb824d888bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:49.368586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-54da2416-3df5-43c4-bd06-1bb824d888bc None None] MFA Rules not processed for user `581d7e60bfdc49a9b492d6c89ae9268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:58:49.369851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-54da2416-3df5-43c4-bd06-1bb824d888bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:49.438157 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1420/2811] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:58:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:58:49.449512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c09e80-944f-4fbb-aed7-af2f8ec00a1b admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:49.451248 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-34c09e80-944f-4fbb-aed7-af2f8ec00a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:49.534853 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c09e80-944f-4fbb-aed7-af2f8ec00a1b admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:49.541639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f3358-18f8-44a3-a3c3-38122fa0c1e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:49.542353 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f3358-18f8-44a3-a3c3-38122fa0c1e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:49.542699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f3358-18f8-44a3-a3c3-38122fa0c1e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:49.543043 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f3358-18f8-44a3-a3c3-38122fa0c1e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:49.552942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f3358-18f8-44a3-a3c3-38122fa0c1e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:49.554582 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f3358-18f8-44a3-a3c3-38122fa0c1e7 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:49.565744 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1392/2812] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:58:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:58:49.590229 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da2416-3df5-43c4-bd06-1bb824d888bc None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:49.662254 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da2416-3df5-43c4-bd06-1bb824d888bc None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:49.666859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:49.667486 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:49.667765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:49.668038 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] PATH_INFO: `/v3/users/581d7e60bfdc49a9b492d6c89ae9268e` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:49.669765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=581d7e60bfdc49a9b492d6c89ae9268e)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:49.671699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:49.674266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:58:49.918749 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '581d7e60bfdc49a9b492d6c89ae9268e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:50.052637 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '581d7e60bfdc49a9b492d6c89ae9268e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:50.053285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '581d7e60bfdc49a9b492d6c89ae9268e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:50.061478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '581d7e60bfdc49a9b492d6c89ae9268e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:50.074395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '581d7e60bfdc49a9b492d6c89ae9268e'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:50.083295 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1421/2813] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:58:49 2022] DELETE /identity/v3/users/581d7e60bfdc49a9b492d6c89ae9268e => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:58:50.088195 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334c2e4-1e4c-4634-ae97-8dc09aa42808 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:50.088477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334c2e4-1e4c-4634-ae97-8dc09aa42808 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:50.088748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334c2e4-1e4c-4634-ae97-8dc09aa42808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:50.109848 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-3334c2e4-1e4c-4634-ae97-8dc09aa42808 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:58:50.111202 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3334c2e4-1e4c-4634-ae97-8dc09aa42808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:50.145105 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:58:50.175302 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1393/2814] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:58:50.185817 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:50.187311 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:50.248927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb36d3d-5907-4754-927b-0e4cfebb8bf5 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:50.252565 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93183ae7-ccae-4ab6-a01a-a2827ddf42c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:50.253156 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93183ae7-ccae-4ab6-a01a-a2827ddf42c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:50.253433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93183ae7-ccae-4ab6-a01a-a2827ddf42c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:50.253725 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93183ae7-ccae-4ab6-a01a-a2827ddf42c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:50.263344 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93183ae7-ccae-4ab6-a01a-a2827ddf42c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:50.265225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93183ae7-ccae-4ab6-a01a-a2827ddf42c7 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:50.275463 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1422/2815] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:58:50.839790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334c2e4-1e4c-4634-ae97-8dc09aa42808 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:50.839790 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-3334c2e4-1e4c-4634-ae97-8dc09aa42808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:50.846330 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334c2e4-1e4c-4634-ae97-8dc09aa42808 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:50.852761 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc2f7a-bf18-467d-b447-5c78528f58c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:50.853640 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc2f7a-bf18-467d-b447-5c78528f58c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:50.854181 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc2f7a-bf18-467d-b447-5c78528f58c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:50.854513 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc2f7a-bf18-467d-b447-5c78528f58c8 admin admin] PATH_INFO: `/v3/projects/973115e67d7749d785795773d18de181` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:50.857444 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc2f7a-bf18-467d-b447-5c78528f58c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=973115e67d7749d785795773d18de181)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:50.860049 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc2f7a-bf18-467d-b447-5c78528f58c8 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:51.036936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2edc2f7a-bf18-467d-b447-5c78528f58c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '973115e67d7749d785795773d18de181', 'request_id': 'req-2edc2f7a-bf18-467d-b447-5c78528f58c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:51.044604 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2edc2f7a-bf18-467d-b447-5c78528f58c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '973115e67d7749d785795773d18de181', 'request_id': 'req-2edc2f7a-bf18-467d-b447-5c78528f58c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:51.050873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2edc2f7a-bf18-467d-b447-5c78528f58c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '973115e67d7749d785795773d18de181', 'request_id': 'req-2edc2f7a-bf18-467d-b447-5c78528f58c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:51.149947 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2edc2f7a-bf18-467d-b447-5c78528f58c8 admin admin] The token cache is being invalidate because project 973115e67d7749d785795773d18de181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:58:51.150278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2edc2f7a-bf18-467d-b447-5c78528f58c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:51.150936 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2edc2f7a-bf18-467d-b447-5c78528f58c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:58:51.152494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1394/2816] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:58:50 2022] DELETE /identity/v3/projects/973115e67d7749d785795773d18de181 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:58:51.157782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feb71c-221c-4bce-a8a4-0bf53396ca82 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:51.158668 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feb71c-221c-4bce-a8a4-0bf53396ca82 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:51.159147 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feb71c-221c-4bce-a8a4-0bf53396ca82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:51.180164 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-32feb71c-221c-4bce-a8a4-0bf53396ca82 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:58:51.181386 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-32feb71c-221c-4bce-a8a4-0bf53396ca82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:51.245417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1423/2817] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:58:51.254873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace95e7-6e3d-47fb-b766-9197aca306a2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:51.255477 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace95e7-6e3d-47fb-b766-9197aca306a2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:51.255602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace95e7-6e3d-47fb-b766-9197aca306a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:51.277489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-1ace95e7-6e3d-47fb-b766-9197aca306a2 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:58:51.279413 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1ace95e7-6e3d-47fb-b766-9197aca306a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:51.289009 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1395/2818] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:58:51.294719 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feb71c-221c-4bce-a8a4-0bf53396ca82 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:51.307572 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feb71c-221c-4bce-a8a4-0bf53396ca82 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:51.314987 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e4b11-4751-4709-972b-d44ceb4cfcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:51.315501 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e4b11-4751-4709-972b-d44ceb4cfcb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:51.315969 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e4b11-4751-4709-972b-d44ceb4cfcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:51.316404 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e4b11-4751-4709-972b-d44ceb4cfcb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:51.317467 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e4b11-4751-4709-972b-d44ceb4cfcb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:58:51.326429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e4b11-4751-4709-972b-d44ceb4cfcb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:51.326429 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e4b11-4751-4709-972b-d44ceb4cfcb1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:51.335844 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1424/2819] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:58:51.340201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace95e7-6e3d-47fb-b766-9197aca306a2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:51.351325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace95e7-6e3d-47fb-b766-9197aca306a2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:51.356100 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6af0ca-fdc9-4198-9af5-84fafe6264ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:51.356705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6af0ca-fdc9-4198-9af5-84fafe6264ff admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:51.356989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6af0ca-fdc9-4198-9af5-84fafe6264ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:51.357264 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6af0ca-fdc9-4198-9af5-84fafe6264ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:51.357986 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6af0ca-fdc9-4198-9af5-84fafe6264ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:51.359952 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6af0ca-fdc9-4198-9af5-84fafe6264ff admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:51.487161 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1396/2820] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:58:51 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:58:51.491189 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e4b11-4751-4709-972b-d44ceb4cfcb1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:51.502006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e4b11-4751-4709-972b-d44ceb4cfcb1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:51.510462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:58:51.561787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8a3a5-2103-4660-8bed-6a795bd3df7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:51.562396 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8a3a5-2103-4660-8bed-6a795bd3df7f admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:51.562677 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8a3a5-2103-4660-8bed-6a795bd3df7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:51.562950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8a3a5-2103-4660-8bed-6a795bd3df7f admin admin] PATH_INFO: `/v3/users` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:51.563569 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8a3a5-2103-4660-8bed-6a795bd3df7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:51.565859 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8a3a5-2103-4660-8bed-6a795bd3df7f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:51.733051 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1425/2821] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:58:51 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:58:51.739914 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6af0ca-fdc9-4198-9af5-84fafe6264ff admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:51.749814 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8a3a5-2103-4660-8bed-6a795bd3df7f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:51.751629 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6af0ca-fdc9-4198-9af5-84fafe6264ff admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:51.751815 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-63a8a3a5-2103-4660-8bed-6a795bd3df7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:51.756485 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fe94e-d8a5-4e55-ba6e-ea0865b37cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:51.757102 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fe94e-d8a5-4e55-ba6e-ea0865b37cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:51.757383 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fe94e-d8a5-4e55-ba6e-ea0865b37cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:51.757703 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fe94e-d8a5-4e55-ba6e-ea0865b37cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:51.759198 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fe94e-d8a5-4e55-ba6e-ea0865b37cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:51.761568 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fe94e-d8a5-4e55-ba6e-ea0865b37cbd admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:51.775831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1397/2822] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:58:51.776845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fe94e-d8a5-4e55-ba6e-ea0865b37cbd admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:51.786865 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fe94e-d8a5-4e55-ba6e-ea0865b37cbd admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:51.791733 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1520623-019b-40d9-84e0-6ccb9783cf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:51.792325 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1520623-019b-40d9-84e0-6ccb9783cf4e admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:51.792610 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1520623-019b-40d9-84e0-6ccb9783cf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:51.792887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1520623-019b-40d9-84e0-6ccb9783cf4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:51.793583 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1520623-019b-40d9-84e0-6ccb9783cf4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:51.795303 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1520623-019b-40d9-84e0-6ccb9783cf4e admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:51.805651 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1398/2823] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:58:51.812053 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8a3a5-2103-4660-8bed-6a795bd3df7f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:51.814851 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1520623-019b-40d9-84e0-6ccb9783cf4e admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:51.816221 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa461aa-261d-4f45-ba34-00cefc2616fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:51.816830 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa461aa-261d-4f45-ba34-00cefc2616fd service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:51.817119 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa461aa-261d-4f45-ba34-00cefc2616fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:51.817380 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa461aa-261d-4f45-ba34-00cefc2616fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:51.819285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ffa461aa-261d-4f45-ba34-00cefc2616fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:51.824525 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1520623-019b-40d9-84e0-6ccb9783cf4e admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:51.830061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777da4b-11df-4a74-8f8c-20c5192f9859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:51.830708 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777da4b-11df-4a74-8f8c-20c5192f9859 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:51.830982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777da4b-11df-4a74-8f8c-20c5192f9859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:51.831250 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777da4b-11df-4a74-8f8c-20c5192f9859 admin admin] PATH_INFO: `/v3/projects/86a19ee530074672936cd7f57f1b4b84/users/4869cb3da4044b1dbc2b4f81a9c24607/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:51.845942 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5777da4b-11df-4a74-8f8c-20c5192f9859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86a19ee530074672936cd7f57f1b4b84, user_id=4869cb3da4044b1dbc2b4f81a9c24607, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:51.848579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5777da4b-11df-4a74-8f8c-20c5192f9859 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:51.890753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa461aa-261d-4f45-ba34-00cefc2616fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:51.892323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa461aa-261d-4f45-ba34-00cefc2616fd service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:51.903881 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1426/2824] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:58:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:58:51.962124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1399/2825] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:58:51 2022] PUT /identity/v3/projects/86a19ee530074672936cd7f57f1b4b84/users/4869cb3da4044b1dbc2b4f81a9c24607/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:58:51.966877 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacdb65-4087-4d6b-b907-0f89a10ca6b5 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:51.967202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacdb65-4087-4d6b-b907-0f89a10ca6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:51.967434 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacdb65-4087-4d6b-b907-0f89a10ca6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:51.995671 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-fdacdb65-4087-4d6b-b907-0f89a10ca6b5 None None] MFA Rules not processed for user `4869cb3da4044b1dbc2b4f81a9c24607`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:58:51.997580 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fdacdb65-4087-4d6b-b907-0f89a10ca6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:52.062849 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1427/2826] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:58:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:58:52.262705 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777da4b-11df-4a74-8f8c-20c5192f9859 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:52.309052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:58:52.320546 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777da4b-11df-4a74-8f8c-20c5192f9859 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:52.324308 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0f23c-86dc-42f3-8aec-2bdb2c1e356e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:52.329223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0f23c-86dc-42f3-8aec-2bdb2c1e356e service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:52.329223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0f23c-86dc-42f3-8aec-2bdb2c1e356e service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:52.329223 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0f23c-86dc-42f3-8aec-2bdb2c1e356e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:52.334201 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c0f23c-86dc-42f3-8aec-2bdb2c1e356e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:52.335699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c0f23c-86dc-42f3-8aec-2bdb2c1e356e service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:52.345200 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1400/2827] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:58:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:58:52.750156 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:58:54.416089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47a25a-53e6-453f-80a0-412c81394918 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:54.416798 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47a25a-53e6-453f-80a0-412c81394918 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:54.416798 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47a25a-53e6-453f-80a0-412c81394918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:54.437715 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-8a47a25a-53e6-453f-80a0-412c81394918 None None] MFA Rules not processed for user `4869cb3da4044b1dbc2b4f81a9c24607`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:58:54.438925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8a47a25a-53e6-453f-80a0-412c81394918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:54.446117 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1428/2828] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:58:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:58:58.788995 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffbdca-3814-4c56-b185-a879dd86e1eb None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:58.789608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffbdca-3814-4c56-b185-a879dd86e1eb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:58.789608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffbdca-3814-4c56-b185-a879dd86e1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:58.812431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9dffbdca-3814-4c56-b185-a879dd86e1eb None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:58:58.813690 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9dffbdca-3814-4c56-b185-a879dd86e1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:58.888571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1401/2829] 10.222.0.57 () {58 vars in 967 bytes} [Tue Aug 9 18:58:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:58:58.898209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47a25a-53e6-453f-80a0-412c81394918 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:58:58.898209 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-8a47a25a-53e6-453f-80a0-412c81394918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:58:58.973271 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47a25a-53e6-453f-80a0-412c81394918 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:58:58.978738 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09f71d-dbea-4308-a719-2cbf0dac8c24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:58:58.979376 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09f71d-dbea-4308-a719-2cbf0dac8c24 service swift] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:58:58.979659 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09f71d-dbea-4308-a719-2cbf0dac8c24 service swift] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:58:58.979936 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09f71d-dbea-4308-a719-2cbf0dac8c24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:58:58.989886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09f71d-dbea-4308-a719-2cbf0dac8c24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:58:58.991547 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09f71d-dbea-4308-a719-2cbf0dac8c24 service swift] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:58:59.001764 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1429/2830] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:58:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 18:59:04.112525 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffbdca-3814-4c56-b185-a879dd86e1eb None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:04.114578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9dffbdca-3814-4c56-b185-a879dd86e1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:04.212486 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffbdca-3814-4c56-b185-a879dd86e1eb None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:04.218615 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b0eee-bda9-4456-ad6a-931e82377b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:04.219338 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b0eee-bda9-4456-ad6a-931e82377b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:04.219626 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b0eee-bda9-4456-ad6a-931e82377b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:04.219899 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b0eee-bda9-4456-ad6a-931e82377b91 admin admin] PATH_INFO: `/v3/users/aa415c4e0e9c4928b51152fa0a428d9b` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:04.221104 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b0eee-bda9-4456-ad6a-931e82377b91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa415c4e0e9c4928b51152fa0a428d9b)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:04.223079 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b0eee-bda9-4456-ad6a-931e82377b91 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:04.467391 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-df9b0eee-bda9-4456-ad6a-931e82377b91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa415c4e0e9c4928b51152fa0a428d9b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:04.572624 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-df9b0eee-bda9-4456-ad6a-931e82377b91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa415c4e0e9c4928b51152fa0a428d9b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:04.573273 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-df9b0eee-bda9-4456-ad6a-931e82377b91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa415c4e0e9c4928b51152fa0a428d9b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:04.583108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-df9b0eee-bda9-4456-ad6a-931e82377b91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa415c4e0e9c4928b51152fa0a428d9b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:04.599641 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-df9b0eee-bda9-4456-ad6a-931e82377b91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa415c4e0e9c4928b51152fa0a428d9b'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:04.607134 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1402/2831] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:59:04 2022] DELETE /identity/v3/users/aa415c4e0e9c4928b51152fa0a428d9b => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:59:04.998592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09f71d-dbea-4308-a719-2cbf0dac8c24 service swift] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:05.014118 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09f71d-dbea-4308-a719-2cbf0dac8c24 service swift] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:05.088650 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a099f-e899-4111-b56b-fd9341864a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:05.089292 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a099f-e899-4111-b56b-fd9341864a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:05.089601 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a099f-e899-4111-b56b-fd9341864a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:05.089879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a099f-e899-4111-b56b-fd9341864a52 admin admin] PATH_INFO: `/v3/projects/7e81a89ed40c43f28ac96cbe3744e6c0` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:05.091564 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a099f-e899-4111-b56b-fd9341864a52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e81a89ed40c43f28ac96cbe3744e6c0)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:05.094088 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a099f-e899-4111-b56b-fd9341864a52 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:05.247365 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-125a099f-e899-4111-b56b-fd9341864a52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e81a89ed40c43f28ac96cbe3744e6c0', 'request_id': 'req-125a099f-e899-4111-b56b-fd9341864a52'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:05.252442 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:05.253744 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-125a099f-e899-4111-b56b-fd9341864a52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e81a89ed40c43f28ac96cbe3744e6c0', 'request_id': 'req-125a099f-e899-4111-b56b-fd9341864a52'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:05.264661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-125a099f-e899-4111-b56b-fd9341864a52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e81a89ed40c43f28ac96cbe3744e6c0', 'request_id': 'req-125a099f-e899-4111-b56b-fd9341864a52'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:05.395084 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-125a099f-e899-4111-b56b-fd9341864a52 admin admin] The token cache is being invalidate because project 7e81a89ed40c43f28ac96cbe3744e6c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:59:05.395397 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-125a099f-e899-4111-b56b-fd9341864a52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:05.396036 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-125a099f-e899-4111-b56b-fd9341864a52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:05.397702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1430/2832] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:59:04 2022] DELETE /identity/v3/projects/7e81a89ed40c43f28ac96cbe3744e6c0 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:59:05.402821 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ed81e-be78-4271-80f3-5957d202bc66 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:05.403370 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ed81e-be78-4271-80f3-5957d202bc66 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:05.403639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ed81e-be78-4271-80f3-5957d202bc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:05.425511 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-fe7ed81e-be78-4271-80f3-5957d202bc66 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:05.426763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-fe7ed81e-be78-4271-80f3-5957d202bc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:05.432313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:05.499471 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1403/2833] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:59:05.507465 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cecf6-b01b-411c-95d5-2a04c10b34ba None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:05.507837 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cecf6-b01b-411c-95d5-2a04c10b34ba None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:05.508022 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cecf6-b01b-411c-95d5-2a04c10b34ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:05.541019 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a98cecf6-b01b-411c-95d5-2a04c10b34ba None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:05.545222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a98cecf6-b01b-411c-95d5-2a04c10b34ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:05.557306 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1431/2834] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:59:05.561813 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ed81e-be78-4271-80f3-5957d202bc66 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:05.577692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ed81e-be78-4271-80f3-5957d202bc66 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:05.582376 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e464cdd-16e1-4a6c-90bf-ece2c5609fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:05.582999 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e464cdd-16e1-4a6c-90bf-ece2c5609fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:05.583283 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e464cdd-16e1-4a6c-90bf-ece2c5609fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:05.583554 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e464cdd-16e1-4a6c-90bf-ece2c5609fd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:05.584396 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e464cdd-16e1-4a6c-90bf-ece2c5609fd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:59:05.584872 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e464cdd-16e1-4a6c-90bf-ece2c5609fd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:05.586507 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e464cdd-16e1-4a6c-90bf-ece2c5609fd2 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:05.598854 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1404/2835] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:59:05.603314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cecf6-b01b-411c-95d5-2a04c10b34ba None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:05.613451 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cecf6-b01b-411c-95d5-2a04c10b34ba None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:05.618556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb08ad-f659-44cb-b6d6-b7c22c8bc9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:05.619160 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb08ad-f659-44cb-b6d6-b7c22c8bc9da admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:05.619438 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb08ad-f659-44cb-b6d6-b7c22c8bc9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:05.619714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb08ad-f659-44cb-b6d6-b7c22c8bc9da admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:05.620787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb08ad-f659-44cb-b6d6-b7c22c8bc9da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:05.622874 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb08ad-f659-44cb-b6d6-b7c22c8bc9da admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:05.735080 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1432/2836] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:59:05 2022] POST /identity/v3/projects => generated 391 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:59:05.739240 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e464cdd-16e1-4a6c-90bf-ece2c5609fd2 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:05.750120 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e464cdd-16e1-4a6c-90bf-ece2c5609fd2 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:05.811578 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e111ab7-9bcd-4bd0-ac18-146671f6cc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:05.812677 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e111ab7-9bcd-4bd0-ac18-146671f6cc69 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:05.813114 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e111ab7-9bcd-4bd0-ac18-146671f6cc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:05.813583 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e111ab7-9bcd-4bd0-ac18-146671f6cc69 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:05.814692 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e111ab7-9bcd-4bd0-ac18-146671f6cc69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:05.816636 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e111ab7-9bcd-4bd0-ac18-146671f6cc69 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:05.927335 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1405/2837] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:59:05 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:59:05.931585 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb08ad-f659-44cb-b6d6-b7c22c8bc9da admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:05.942358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb08ad-f659-44cb-b6d6-b7c22c8bc9da admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:05.947278 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752d9f2-4ae4-40b6-a0e0-c5f03ae3b263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:05.947945 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752d9f2-4ae4-40b6-a0e0-c5f03ae3b263 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:05.948249 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752d9f2-4ae4-40b6-a0e0-c5f03ae3b263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:05.948524 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752d9f2-4ae4-40b6-a0e0-c5f03ae3b263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:05.949164 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752d9f2-4ae4-40b6-a0e0-c5f03ae3b263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:05.950827 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752d9f2-4ae4-40b6-a0e0-c5f03ae3b263 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:05.961613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1433/2838] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:59:05.965989 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e111ab7-9bcd-4bd0-ac18-146671f6cc69 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:05.976225 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e111ab7-9bcd-4bd0-ac18-146671f6cc69 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:05.980946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a0233-623d-4859-a65d-6edd7a21c9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:05.981608 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a0233-623d-4859-a65d-6edd7a21c9ec admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:05.982406 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a0233-623d-4859-a65d-6edd7a21c9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:05.982949 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a0233-623d-4859-a65d-6edd7a21c9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:05.983602 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a0233-623d-4859-a65d-6edd7a21c9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:05.985191 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a0233-623d-4859-a65d-6edd7a21c9ec admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:05.995982 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1406/2839] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:59:06.000328 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752d9f2-4ae4-40b6-a0e0-c5f03ae3b263 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:06.021006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752d9f2-4ae4-40b6-a0e0-c5f03ae3b263 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:06.026056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441db33-758f-41f2-b377-41e21ad42440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:06.026700 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441db33-758f-41f2-b377-41e21ad42440 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:06.027042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441db33-758f-41f2-b377-41e21ad42440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:06.027321 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441db33-758f-41f2-b377-41e21ad42440 admin admin] PATH_INFO: `/v3/projects/e5988bcb057b4797a7f7df7c38b3aeab/users/08a04adb12d444ce95e3c76bf13d7b61/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:06.045781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441db33-758f-41f2-b377-41e21ad42440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5988bcb057b4797a7f7df7c38b3aeab, user_id=08a04adb12d444ce95e3c76bf13d7b61, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:06.048475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441db33-758f-41f2-b377-41e21ad42440 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:06.145632 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1434/2840] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:59:05 2022] PUT /identity/v3/projects/e5988bcb057b4797a7f7df7c38b3aeab/users/08a04adb12d444ce95e3c76bf13d7b61/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:59:06.156732 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e1b67-fd8d-40be-b8de-c8bcadd83864 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:06.157018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e1b67-fd8d-40be-b8de-c8bcadd83864 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:06.159052 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e1b67-fd8d-40be-b8de-c8bcadd83864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:06.193962 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4c7e1b67-fd8d-40be-b8de-c8bcadd83864 None None] MFA Rules not processed for user `08a04adb12d444ce95e3c76bf13d7b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:06.194990 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4c7e1b67-fd8d-40be-b8de-c8bcadd83864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:06.282392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1407/2841] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:59:06.291098 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441db33-758f-41f2-b377-41e21ad42440 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:06.293045 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-6441db33-758f-41f2-b377-41e21ad42440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:06.362739 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441db33-758f-41f2-b377-41e21ad42440 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:06.366462 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbb6eb-bc50-4db6-90af-022367bec749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:06.367075 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbb6eb-bc50-4db6-90af-022367bec749 service cinder] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:06.367351 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbb6eb-bc50-4db6-90af-022367bec749 service cinder] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:06.372264 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbb6eb-bc50-4db6-90af-022367bec749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:06.377617 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbb6eb-bc50-4db6-90af-022367bec749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:06.379223 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbb6eb-bc50-4db6-90af-022367bec749 service cinder] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:06.394908 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1435/2842] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:59:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:59:09.368922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e1b67-fd8d-40be-b8de-c8bcadd83864 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:09.370439 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4c7e1b67-fd8d-40be-b8de-c8bcadd83864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:09.382933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:09.439133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e1b67-fd8d-40be-b8de-c8bcadd83864 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:09.443963 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:09.444639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:09.444920 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:09.445196 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] PATH_INFO: `/v3/users/db26f48da0af4723afb2c0a4c5f52d07` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:09.446367 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] RBAC: Authorizing `identity:delete_user(user_id=db26f48da0af4723afb2c0a4c5f52d07)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:09.448277 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:09.654975 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db26f48da0af4723afb2c0a4c5f52d07'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:09.726612 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db26f48da0af4723afb2c0a4c5f52d07'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:09.727258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db26f48da0af4723afb2c0a4c5f52d07'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:09.734807 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db26f48da0af4723afb2c0a4c5f52d07'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:09.746979 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db26f48da0af4723afb2c0a4c5f52d07'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:09.753512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1408/2843] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:59:09 2022] DELETE /identity/v3/users/db26f48da0af4723afb2c0a4c5f52d07 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:59:10.026932 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbb6eb-bc50-4db6-90af-022367bec749 service cinder] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:10.038917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbb6eb-bc50-4db6-90af-022367bec749 service cinder] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:10.101425 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:10.102089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:10.102368 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:10.102639 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] PATH_INFO: `/v3/projects/6bedacf1dd4e4398ae3acefcd58a93a8` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:10.103787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bedacf1dd4e4398ae3acefcd58a93a8)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:10.105672 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:10.236613 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:10.252302 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bedacf1dd4e4398ae3acefcd58a93a8', 'request_id': 'req-fea577c9-3e77-492e-bb6e-51fd50493ae1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:10.257574 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bedacf1dd4e4398ae3acefcd58a93a8', 'request_id': 'req-fea577c9-3e77-492e-bb6e-51fd50493ae1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:10.264277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bedacf1dd4e4398ae3acefcd58a93a8', 'request_id': 'req-fea577c9-3e77-492e-bb6e-51fd50493ae1'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:10.366781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] The token cache is being invalidate because project 6bedacf1dd4e4398ae3acefcd58a93a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:59:10.367078 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:10.367662 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:10.369125 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1436/2844] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:59:10 2022] DELETE /identity/v3/projects/6bedacf1dd4e4398ae3acefcd58a93a8 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:59:10.373740 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:10.375148 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:10.388820 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4370d-f857-491d-ac68-87456b9cdadb admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:10.446946 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890fd0e-510b-4838-88ee-c1134da55927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:10.447594 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890fd0e-510b-4838-88ee-c1134da55927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:10.447882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890fd0e-510b-4838-88ee-c1134da55927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:10.448169 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890fd0e-510b-4838-88ee-c1134da55927 admin admin] PATH_INFO: `/v3/users/4cd1e14552324128b60c2d188a33f8c8` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:10.449340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890fd0e-510b-4838-88ee-c1134da55927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cd1e14552324128b60c2d188a33f8c8)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:10.451933 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890fd0e-510b-4838-88ee-c1134da55927 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:10.546809 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:10.665133 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2890fd0e-510b-4838-88ee-c1134da55927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cd1e14552324128b60c2d188a33f8c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:10.749467 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2890fd0e-510b-4838-88ee-c1134da55927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cd1e14552324128b60c2d188a33f8c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:10.750122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2890fd0e-510b-4838-88ee-c1134da55927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cd1e14552324128b60c2d188a33f8c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:10.757455 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2890fd0e-510b-4838-88ee-c1134da55927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cd1e14552324128b60c2d188a33f8c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:10.768889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-2890fd0e-510b-4838-88ee-c1134da55927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cd1e14552324128b60c2d188a33f8c8'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:10.774907 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1409/2845] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:59:10 2022] DELETE /identity/v3/users/4cd1e14552324128b60c2d188a33f8c8 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:59:11.008440 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:11.009977 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:11.032988 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea577c9-3e77-492e-bb6e-51fd50493ae1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:11.090563 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f1fb2-035b-466f-8596-a97aba65159c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:11.091811 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f1fb2-035b-466f-8596-a97aba65159c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:11.092085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f1fb2-035b-466f-8596-a97aba65159c admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:11.092361 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f1fb2-035b-466f-8596-a97aba65159c admin admin] PATH_INFO: `/v3/projects/76bde2e6dfc040378bbc58faca557604` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:11.093478 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f1fb2-035b-466f-8596-a97aba65159c admin admin] RBAC: Authorizing `identity:delete_project(project_id=76bde2e6dfc040378bbc58faca557604)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:11.095436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f1fb2-035b-466f-8596-a97aba65159c admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:11.224751 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a2f1fb2-035b-466f-8596-a97aba65159c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76bde2e6dfc040378bbc58faca557604', 'request_id': 'req-0a2f1fb2-035b-466f-8596-a97aba65159c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:11.230176 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a2f1fb2-035b-466f-8596-a97aba65159c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76bde2e6dfc040378bbc58faca557604', 'request_id': 'req-0a2f1fb2-035b-466f-8596-a97aba65159c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:11.236973 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a2f1fb2-035b-466f-8596-a97aba65159c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76bde2e6dfc040378bbc58faca557604', 'request_id': 'req-0a2f1fb2-035b-466f-8596-a97aba65159c'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:11.321018 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a2f1fb2-035b-466f-8596-a97aba65159c admin admin] The token cache is being invalidate because project 76bde2e6dfc040378bbc58faca557604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:59:11.321310 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a2f1fb2-035b-466f-8596-a97aba65159c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:11.321960 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-0a2f1fb2-035b-466f-8596-a97aba65159c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:11.323538 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1437/2846] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:59:11 2022] DELETE /identity/v3/projects/76bde2e6dfc040378bbc58faca557604 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:59:11.329151 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d5b83-179d-4a22-9620-15bd8ab90683 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:11.329440 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d5b83-179d-4a22-9620-15bd8ab90683 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:11.329774 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d5b83-179d-4a22-9620-15bd8ab90683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:11.350827 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-e53d5b83-179d-4a22-9620-15bd8ab90683 None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:11.353060 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-e53d5b83-179d-4a22-9620-15bd8ab90683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:11.412631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1410/2847] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:59:11.421106 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429bae1-4aac-48ed-a12a-1429e4621fef None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:11.421388 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429bae1-4aac-48ed-a12a-1429e4621fef None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:11.421693 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429bae1-4aac-48ed-a12a-1429e4621fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:11.441403 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-5429bae1-4aac-48ed-a12a-1429e4621fef None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:11.442626 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5429bae1-4aac-48ed-a12a-1429e4621fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:11.450991 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1438/2848] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:59:11.455098 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d5b83-179d-4a22-9620-15bd8ab90683 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:11.464639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d5b83-179d-4a22-9620-15bd8ab90683 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:11.469816 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215df6d-40c0-4c3e-a1cc-3ef838042461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:11.470400 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215df6d-40c0-4c3e-a1cc-3ef838042461 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:11.470677 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215df6d-40c0-4c3e-a1cc-3ef838042461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:11.470948 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215df6d-40c0-4c3e-a1cc-3ef838042461 admin admin] PATH_INFO: `/v3/domains` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:11.471618 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215df6d-40c0-4c3e-a1cc-3ef838042461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:59:11.471917 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215df6d-40c0-4c3e-a1cc-3ef838042461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:11.473489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215df6d-40c0-4c3e-a1cc-3ef838042461 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:11.485388 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1411/2849] 10.222.0.57 () {58 vars in 1153 bytes} [Tue Aug 9 18:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:59:11.489270 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429bae1-4aac-48ed-a12a-1429e4621fef None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:11.498545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429bae1-4aac-48ed-a12a-1429e4621fef None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:11.503219 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40254f27-6aab-4c33-bf4a-fbae21916952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:11.503803 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40254f27-6aab-4c33-bf4a-fbae21916952 admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:11.504083 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40254f27-6aab-4c33-bf4a-fbae21916952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:11.504356 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40254f27-6aab-4c33-bf4a-fbae21916952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:11.504971 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40254f27-6aab-4c33-bf4a-fbae21916952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:11.506905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40254f27-6aab-4c33-bf4a-fbae21916952 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:11.602305 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1439/2850] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:59:11 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:59:11.606538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215df6d-40c0-4c3e-a1cc-3ef838042461 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:11.616911 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215df6d-40c0-4c3e-a1cc-3ef838042461 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:11.673792 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72821831-61a2-410a-bcaf-ff1f3ba5f8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:11.674392 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72821831-61a2-410a-bcaf-ff1f3ba5f8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:11.674675 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72821831-61a2-410a-bcaf-ff1f3ba5f8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:11.674953 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72821831-61a2-410a-bcaf-ff1f3ba5f8d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:11.675566 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72821831-61a2-410a-bcaf-ff1f3ba5f8d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:11.677463 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72821831-61a2-410a-bcaf-ff1f3ba5f8d9 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:11.773179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1412/2851] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:59:11 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:59:11.777068 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40254f27-6aab-4c33-bf4a-fbae21916952 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:11.788201 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40254f27-6aab-4c33-bf4a-fbae21916952 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:11.792782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27992190-b7ae-4243-bfae-9b9c2a11a74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:11.793399 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27992190-b7ae-4243-bfae-9b9c2a11a74d admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:11.793753 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27992190-b7ae-4243-bfae-9b9c2a11a74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:11.794014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27992190-b7ae-4243-bfae-9b9c2a11a74d admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:11.794658 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27992190-b7ae-4243-bfae-9b9c2a11a74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:11.796285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27992190-b7ae-4243-bfae-9b9c2a11a74d admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:11.810011 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1440/2852] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:59:11.811042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72821831-61a2-410a-bcaf-ff1f3ba5f8d9 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:11.821550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72821831-61a2-410a-bcaf-ff1f3ba5f8d9 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:11.826697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85a051-2618-4883-9b58-9804abf2ee17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:11.827398 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85a051-2618-4883-9b58-9804abf2ee17 admin admin] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:11.827709 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85a051-2618-4883-9b58-9804abf2ee17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:11.827983 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85a051-2618-4883-9b58-9804abf2ee17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:11.828631 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85a051-2618-4883-9b58-9804abf2ee17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:11.830519 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85a051-2618-4883-9b58-9804abf2ee17 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:11.841992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1413/2853] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:59:11.846260 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27992190-b7ae-4243-bfae-9b9c2a11a74d admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:11.853475 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:11.858606 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27992190-b7ae-4243-bfae-9b9c2a11a74d admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:11.864266 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e9b60-8dbb-4bee-9e4f-b17f4ee786d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:11.864944 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e9b60-8dbb-4bee-9e4f-b17f4ee786d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:11.865226 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e9b60-8dbb-4bee-9e4f-b17f4ee786d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:11.865500 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e9b60-8dbb-4bee-9e4f-b17f4ee786d4 admin admin] PATH_INFO: `/v3/projects/098c57b11d7948c8a1ef732af98ec75f/users/dbed10f27ab548809755b32d0456d1bc/roles/08a7c861d84b43d7a4da431f31f3c2d7` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:11.881261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e9b60-8dbb-4bee-9e4f-b17f4ee786d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=098c57b11d7948c8a1ef732af98ec75f, user_id=dbed10f27ab548809755b32d0456d1bc, role_id=08a7c861d84b43d7a4da431f31f3c2d7)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:11.883963 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e9b60-8dbb-4bee-9e4f-b17f4ee786d4 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:12.002161 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1441/2854] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:59:11 2022] PUT /identity/v3/projects/098c57b11d7948c8a1ef732af98ec75f/users/dbed10f27ab548809755b32d0456d1bc/roles/08a7c861d84b43d7a4da431f31f3c2d7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:59:12.007455 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b25cf-dc08-4a28-8960-44b2f7d8ea8d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:12.007739 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b25cf-dc08-4a28-8960-44b2f7d8ea8d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:12.008024 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b25cf-dc08-4a28-8960-44b2f7d8ea8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:12.027158 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-9a3b25cf-dc08-4a28-8960-44b2f7d8ea8d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:12.028340 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-9a3b25cf-dc08-4a28-8960-44b2f7d8ea8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:12.035287 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:12.090093 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1414/2855] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:59:12.099792 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e9b60-8dbb-4bee-9e4f-b17f4ee786d4 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:12.101586 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-489e9b60-8dbb-4bee-9e4f-b17f4ee786d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:12.159282 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e9b60-8dbb-4bee-9e4f-b17f4ee786d4 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:12.162950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32253994-b4b8-4722-91ef-4271ab53533a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:12.163558 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32253994-b4b8-4722-91ef-4271ab53533a service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:12.163840 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32253994-b4b8-4722-91ef-4271ab53533a service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:12.164113 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32253994-b4b8-4722-91ef-4271ab53533a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:12.173285 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32253994-b4b8-4722-91ef-4271ab53533a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:12.174770 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32253994-b4b8-4722-91ef-4271ab53533a service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:12.184298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1442/2856] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:59:18.141556 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf82da-0bb9-448c-991d-5b036d268adb None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:18.142660 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf82da-0bb9-448c-991d-5b036d268adb None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:18.143122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf82da-0bb9-448c-991d-5b036d268adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:18.173291 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4fbf82da-0bb9-448c-991d-5b036d268adb None None] MFA Rules not processed for user `dbed10f27ab548809755b32d0456d1bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:18.174571 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4fbf82da-0bb9-448c-991d-5b036d268adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:18.258921 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1415/2857] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:59:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:59:18.262074 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32253994-b4b8-4722-91ef-4271ab53533a service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:18.272649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32253994-b4b8-4722-91ef-4271ab53533a service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:18.277865 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe03798-5adb-41a4-a7dc-86d2d2a8a8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:18.281794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe03798-5adb-41a4-a7dc-86d2d2a8a8fe admin admin] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:18.281794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe03798-5adb-41a4-a7dc-86d2d2a8a8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:18.281794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe03798-5adb-41a4-a7dc-86d2d2a8a8fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:18.281794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe03798-5adb-41a4-a7dc-86d2d2a8a8fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:18.281794 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe03798-5adb-41a4-a7dc-86d2d2a8a8fe admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:18.374423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1443/2858] 10.222.0.57 () {60 vars in 1153 bytes} [Tue Aug 9 18:59:18 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:59:18.378489 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf82da-0bb9-448c-991d-5b036d268adb None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:18.389806 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf82da-0bb9-448c-991d-5b036d268adb None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:18.454748 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29170ae9-4cf7-4610-be7c-9921abd35046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:18.455356 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29170ae9-4cf7-4610-be7c-9921abd35046 admin admin] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:18.455639 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29170ae9-4cf7-4610-be7c-9921abd35046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:18.455911 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29170ae9-4cf7-4610-be7c-9921abd35046 admin admin] PATH_INFO: `/v3/users` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:18.456550 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29170ae9-4cf7-4610-be7c-9921abd35046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:18.458495 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29170ae9-4cf7-4610-be7c-9921abd35046 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:18.550122 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1416/2859] 10.222.0.57 () {60 vars in 1144 bytes} [Tue Aug 9 18:59:18 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:59:18.558791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe03798-5adb-41a4-a7dc-86d2d2a8a8fe admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:18.570138 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe03798-5adb-41a4-a7dc-86d2d2a8a8fe admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:18.574938 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af967e5-260b-4bf1-b9bf-a0f81782a5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:18.575455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af967e5-260b-4bf1-b9bf-a0f81782a5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:18.575741 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af967e5-260b-4bf1-b9bf-a0f81782a5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:18.576014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af967e5-260b-4bf1-b9bf-a0f81782a5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:18.576688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af967e5-260b-4bf1-b9bf-a0f81782a5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:18.578331 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af967e5-260b-4bf1-b9bf-a0f81782a5e1 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:18.589006 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1444/2860] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:59:18.593704 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29170ae9-4cf7-4610-be7c-9921abd35046 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:18.603595 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29170ae9-4cf7-4610-be7c-9921abd35046 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:18.609203 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baa61c-1c3e-45c7-87b6-d1aee81afe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:18.609879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baa61c-1c3e-45c7-87b6-d1aee81afe71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:18.610156 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baa61c-1c3e-45c7-87b6-d1aee81afe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:18.610428 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baa61c-1c3e-45c7-87b6-d1aee81afe71 admin admin] PATH_INFO: `/v3/projects/42899db38b484d4283e223a722029c3a/users/dc87fff1029748a784868170411309ca/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:18.627362 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baa61c-1c3e-45c7-87b6-d1aee81afe71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42899db38b484d4283e223a722029c3a, user_id=dc87fff1029748a784868170411309ca, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:18.630348 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baa61c-1c3e-45c7-87b6-d1aee81afe71 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:18.709494 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1417/2861] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:59:18 2022] PUT /identity/v3/projects/42899db38b484d4283e223a722029c3a/users/dc87fff1029748a784868170411309ca/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:59:18.713439 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af967e5-260b-4bf1-b9bf-a0f81782a5e1 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:18.724217 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af967e5-260b-4bf1-b9bf-a0f81782a5e1 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:18.781031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2874aa-88fe-466b-97e7-970a0b3b9a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:18.781661 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2874aa-88fe-466b-97e7-970a0b3b9a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:18.781939 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2874aa-88fe-466b-97e7-970a0b3b9a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:18.782207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2874aa-88fe-466b-97e7-970a0b3b9a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:18.782845 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2874aa-88fe-466b-97e7-970a0b3b9a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:18.784433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2874aa-88fe-466b-97e7-970a0b3b9a3f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:18.794866 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1445/2862] 10.222.0.57 () {58 vars in 1122 bytes} [Tue Aug 9 18:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:59:18.799278 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baa61c-1c3e-45c7-87b6-d1aee81afe71 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:18.809891 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baa61c-1c3e-45c7-87b6-d1aee81afe71 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:18.816118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee407677-afe4-4a1f-a55f-4ab137bcfee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:18.816838 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee407677-afe4-4a1f-a55f-4ab137bcfee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:18.817117 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee407677-afe4-4a1f-a55f-4ab137bcfee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:18.826044 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee407677-afe4-4a1f-a55f-4ab137bcfee3 admin admin] PATH_INFO: `/v3/projects/42899db38b484d4283e223a722029c3a/users/dc87fff1029748a784868170411309ca/roles/243b2ac105fc4306afd7108a6cdb0982` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:18.828292 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee407677-afe4-4a1f-a55f-4ab137bcfee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42899db38b484d4283e223a722029c3a, user_id=dc87fff1029748a784868170411309ca, role_id=243b2ac105fc4306afd7108a6cdb0982)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:18.831711 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee407677-afe4-4a1f-a55f-4ab137bcfee3 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:18.834306 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:18.843581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1418/2863] 10.222.0.57 () {60 vars in 1483 bytes} [Tue Aug 9 18:59:18 2022] PUT /identity/v3/projects/42899db38b484d4283e223a722029c3a/users/dc87fff1029748a784868170411309ca/roles/243b2ac105fc4306afd7108a6cdb0982 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:59:18.950056 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e65349-2dc4-49ff-a249-0749b1b53d5a None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:18.950834 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e65349-2dc4-49ff-a249-0749b1b53d5a None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:18.951112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e65349-2dc4-49ff-a249-0749b1b53d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:19.006207 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-c5e65349-2dc4-49ff-a249-0749b1b53d5a None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:19.007457 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-c5e65349-2dc4-49ff-a249-0749b1b53d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:19.028850 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:19.081946 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1446/2864] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:59:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:59:19.094403 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee407677-afe4-4a1f-a55f-4ab137bcfee3 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:19.096216 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ee407677-afe4-4a1f-a55f-4ab137bcfee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:19.192061 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee407677-afe4-4a1f-a55f-4ab137bcfee3 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:19.200839 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fd8b5-0256-4a18-9741-6ad925fd71bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:19.201902 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fd8b5-0256-4a18-9741-6ad925fd71bd service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:19.202493 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fd8b5-0256-4a18-9741-6ad925fd71bd service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:19.202493 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fd8b5-0256-4a18-9741-6ad925fd71bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:19.214124 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fd8b5-0256-4a18-9741-6ad925fd71bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:19.220710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fd8b5-0256-4a18-9741-6ad925fd71bd service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:19.226992 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1419/2865] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:59:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:59:20.339417 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b7be6-55e4-442c-a719-f3ba15f8bb14 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:20.339958 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b7be6-55e4-442c-a719-f3ba15f8bb14 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:20.339958 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b7be6-55e4-442c-a719-f3ba15f8bb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:20.360156 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-dc9b7be6-55e4-442c-a719-f3ba15f8bb14 None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:20.361352 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-dc9b7be6-55e4-442c-a719-f3ba15f8bb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:20.368612 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1447/2866] 10.222.0.57 () {58 vars in 967 bytes} [Tue Aug 9 18:59:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:59:20.383841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fd8b5-0256-4a18-9741-6ad925fd71bd service swift] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:20.396027 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fd8b5-0256-4a18-9741-6ad925fd71bd service swift] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:20.402108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a9af8-78f4-4df8-80d1-580b8c06c42a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:20.403943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9af8-78f4-4df8-80d1-580b8c06c42a service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:20.403943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9af8-78f4-4df8-80d1-580b8c06c42a service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:20.403943 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9af8-78f4-4df8-80d1-580b8c06c42a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:20.413167 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a9af8-78f4-4df8-80d1-580b8c06c42a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:20.414785 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a9af8-78f4-4df8-80d1-580b8c06c42a service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:20.451058 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1420/2867] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:59:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:59:21.163102 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770b252-1423-46a8-9e93-4bd0ee9acd2b None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:21.163389 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770b252-1423-46a8-9e93-4bd0ee9acd2b None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:21.163886 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770b252-1423-46a8-9e93-4bd0ee9acd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:21.183688 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-1770b252-1423-46a8-9e93-4bd0ee9acd2b None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:21.185395 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-1770b252-1423-46a8-9e93-4bd0ee9acd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:21.194436 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1448/2868] 10.222.0.57 () {58 vars in 1009 bytes} [Tue Aug 9 18:59:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:59:21.213000 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a9af8-78f4-4df8-80d1-580b8c06c42a service swift] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:21.224849 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a9af8-78f4-4df8-80d1-580b8c06c42a service swift] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:21.232749 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178df4bc-f528-4c07-aed7-4020355c469e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:21.233369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178df4bc-f528-4c07-aed7-4020355c469e service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:21.233722 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178df4bc-f528-4c07-aed7-4020355c469e service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:21.234229 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178df4bc-f528-4c07-aed7-4020355c469e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:21.246766 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178df4bc-f528-4c07-aed7-4020355c469e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:21.249728 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178df4bc-f528-4c07-aed7-4020355c469e service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:21.259951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1421/2869] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:59:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:59:22.098913 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbd005-50fb-46c7-95dc-f4445062c0a1 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:22.100390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbd005-50fb-46c7-95dc-f4445062c0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:22.100879 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbd005-50fb-46c7-95dc-f4445062c0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:22.134084 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-27cbd005-50fb-46c7-95dc-f4445062c0a1 None None] MFA Rules not processed for user `1255cb6e360244ba9d5cec5cc65a1f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:22.135649 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-27cbd005-50fb-46c7-95dc-f4445062c0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:22.144218 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1449/2870] 10.222.0.57 () {58 vars in 967 bytes} [Tue Aug 9 18:59:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:59:22.162084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178df4bc-f528-4c07-aed7-4020355c469e service swift] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:22.172299 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178df4bc-f528-4c07-aed7-4020355c469e service swift] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:22.179884 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe26190-7999-4cb8-aff0-3e7e7d224e04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff735ecfcdf49089b5cfeafc56048cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:22.180948 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe26190-7999-4cb8-aff0-3e7e7d224e04 service swift] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:22.181395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe26190-7999-4cb8-aff0-3e7e7d224e04 service swift] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:22.182395 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe26190-7999-4cb8-aff0-3e7e7d224e04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:22.194951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe26190-7999-4cb8-aff0-3e7e7d224e04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:22.197378 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe26190-7999-4cb8-aff0-3e7e7d224e04 service swift] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:22.210166 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1422/2871] 10.222.0.57 () {60 vars in 1384 bytes} [Tue Aug 9 18:59:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:59:25.243699 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162700ef-54a9-4d05-a788-a6d447b7ab54 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:25.244187 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162700ef-54a9-4d05-a788-a6d447b7ab54 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:25.244288 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162700ef-54a9-4d05-a788-a6d447b7ab54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:25.281009 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-162700ef-54a9-4d05-a788-a6d447b7ab54 None None] MFA Rules not processed for user `dc87fff1029748a784868170411309ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:25.282235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-162700ef-54a9-4d05-a788-a6d447b7ab54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:25.362950 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1450/2872] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:59:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:59:25.378118 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe26190-7999-4cb8-aff0-3e7e7d224e04 service swift] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:25.380126 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-dbe26190-7999-4cb8-aff0-3e7e7d224e04 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:25.458808 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe26190-7999-4cb8-aff0-3e7e7d224e04 service swift] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:25.464085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162700ef-54a9-4d05-a788-a6d447b7ab54 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:25.464826 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2923e8e-0fe4-4213-b6dd-81597ee3a4ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:25.465438 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2923e8e-0fe4-4213-b6dd-81597ee3a4ff service nova] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:25.466282 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2923e8e-0fe4-4213-b6dd-81597ee3a4ff service nova] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:25.466400 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-162700ef-54a9-4d05-a788-a6d447b7ab54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:25.466621 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2923e8e-0fe4-4213-b6dd-81597ee3a4ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:25.488681 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:25.540039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2923e8e-0fe4-4213-b6dd-81597ee3a4ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:25.541581 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2923e8e-0fe4-4213-b6dd-81597ee3a4ff service nova] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:25.542358 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162700ef-54a9-4d05-a788-a6d447b7ab54 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:25.553497 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1423/2873] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:59:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:59:25.554064 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:25.555726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:25.555935 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:25.556237 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] PATH_INFO: `/v3/users/4869cb3da4044b1dbc2b4f81a9c24607` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:25.557455 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4869cb3da4044b1dbc2b4f81a9c24607)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:25.563779 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:25.682867 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:25.795981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4869cb3da4044b1dbc2b4f81a9c24607'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:25.891856 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4869cb3da4044b1dbc2b4f81a9c24607'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:25.893702 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4869cb3da4044b1dbc2b4f81a9c24607'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:25.907492 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4869cb3da4044b1dbc2b4f81a9c24607'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:25.931527 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4869cb3da4044b1dbc2b4f81a9c24607'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:25.942736 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1451/2874] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:59:25 2022] DELETE /identity/v3/users/4869cb3da4044b1dbc2b4f81a9c24607 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:59:25.957833 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ca346-8cfd-45b3-b311-24a28553d2ad None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:25.958727 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ca346-8cfd-45b3-b311-24a28553d2ad None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:25.958863 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ca346-8cfd-45b3-b311-24a28553d2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:25.981607 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-4b2ca346-8cfd-45b3-b311-24a28553d2ad None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:25.984288 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4b2ca346-8cfd-45b3-b311-24a28553d2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:26.060351 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1424/2875] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:59:26.070810 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:26.072669 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:26.137112 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec1d26-43d0-4904-aa0f-b105141f138e admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:26.142279 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7189675-d691-422d-9d28-8a10f452afd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:26.142901 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7189675-d691-422d-9d28-8a10f452afd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:26.143225 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7189675-d691-422d-9d28-8a10f452afd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:26.143535 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7189675-d691-422d-9d28-8a10f452afd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:26.154002 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7189675-d691-422d-9d28-8a10f452afd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:26.155471 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7189675-d691-422d-9d28-8a10f452afd9 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:26.167398 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1452/2876] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:59:26.454600 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ca346-8cfd-45b3-b311-24a28553d2ad None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:26.456046 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-4b2ca346-8cfd-45b3-b311-24a28553d2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:26.468679 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ca346-8cfd-45b3-b311-24a28553d2ad None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:26.481029 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddb73f-85e1-4967-9def-5fbe80839cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:26.482779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddb73f-85e1-4967-9def-5fbe80839cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:26.483113 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddb73f-85e1-4967-9def-5fbe80839cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:26.483643 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddb73f-85e1-4967-9def-5fbe80839cd0 admin admin] PATH_INFO: `/v3/projects/86a19ee530074672936cd7f57f1b4b84` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:26.484949 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cddb73f-85e1-4967-9def-5fbe80839cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86a19ee530074672936cd7f57f1b4b84)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:26.487108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cddb73f-85e1-4967-9def-5fbe80839cd0 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:26.608791 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6cddb73f-85e1-4967-9def-5fbe80839cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86a19ee530074672936cd7f57f1b4b84', 'request_id': 'req-6cddb73f-85e1-4967-9def-5fbe80839cd0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:26.615447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6cddb73f-85e1-4967-9def-5fbe80839cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86a19ee530074672936cd7f57f1b4b84', 'request_id': 'req-6cddb73f-85e1-4967-9def-5fbe80839cd0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:26.621039 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6cddb73f-85e1-4967-9def-5fbe80839cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86a19ee530074672936cd7f57f1b4b84', 'request_id': 'req-6cddb73f-85e1-4967-9def-5fbe80839cd0'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:26.721699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6cddb73f-85e1-4967-9def-5fbe80839cd0 admin admin] The token cache is being invalidate because project 86a19ee530074672936cd7f57f1b4b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:59:26.722087 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6cddb73f-85e1-4967-9def-5fbe80839cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:26.722820 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-6cddb73f-85e1-4967-9def-5fbe80839cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:26.738416 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1425/2877] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:59:26 2022] DELETE /identity/v3/projects/86a19ee530074672936cd7f57f1b4b84 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:59:28.346309 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7189675-d691-422d-9d28-8a10f452afd9 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:28.347598 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f7189675-d691-422d-9d28-8a10f452afd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:28.368833 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7189675-d691-422d-9d28-8a10f452afd9 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:28.442791 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:28.443431 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:28.443782 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:28.444058 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] PATH_INFO: `/v3/users/08a04adb12d444ce95e3c76bf13d7b61` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:28.445261 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] RBAC: Authorizing `identity:delete_user(user_id=08a04adb12d444ce95e3c76bf13d7b61)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:28.447592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:28.686382 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08a04adb12d444ce95e3c76bf13d7b61'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:28.688801 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:29.041216 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08a04adb12d444ce95e3c76bf13d7b61'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:29.048536 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08a04adb12d444ce95e3c76bf13d7b61'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:29.051966 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08a04adb12d444ce95e3c76bf13d7b61'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:29.065222 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08a04adb12d444ce95e3c76bf13d7b61'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:29.071665 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1453/2878] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 18:59:28 2022] DELETE /identity/v3/users/08a04adb12d444ce95e3c76bf13d7b61 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:59:29.076666 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fa60c-0018-49a7-a0bc-3d14a13b203d None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:29.077725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fa60c-0018-49a7-a0bc-3d14a13b203d None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:29.078470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fa60c-0018-49a7-a0bc-3d14a13b203d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:29.099446 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-ff6fa60c-0018-49a7-a0bc-3d14a13b203d None None] MFA Rules not processed for user `7a379f0e1f1f4c9c884081f682310ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:29.100676 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ff6fa60c-0018-49a7-a0bc-3d14a13b203d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:29.149669 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:29.168802 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1426/2879] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:59:29.189277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:29.189277 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:29.239925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc68038-d5d3-4451-8e6a-2b67a241666f admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:29.243555 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef1337-c0c4-4c2d-ae4c-993837ba3204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:29.244158 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef1337-c0c4-4c2d-ae4c-993837ba3204 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:29.244439 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef1337-c0c4-4c2d-ae4c-993837ba3204 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:29.244714 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef1337-c0c4-4c2d-ae4c-993837ba3204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:29.254317 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef1337-c0c4-4c2d-ae4c-993837ba3204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:29.255748 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef1337-c0c4-4c2d-ae4c-993837ba3204 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:29.268480 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1454/2880] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:59:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:59:29.724652 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fa60c-0018-49a7-a0bc-3d14a13b203d None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:29.726351 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ff6fa60c-0018-49a7-a0bc-3d14a13b203d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:29.741551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fa60c-0018-49a7-a0bc-3d14a13b203d None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:29.747523 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:29.748590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:29.748590 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:29.748831 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] PATH_INFO: `/v3/projects/e5988bcb057b4797a7f7df7c38b3aeab` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:29.750199 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5988bcb057b4797a7f7df7c38b3aeab)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:29.759563 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:29.899559 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5988bcb057b4797a7f7df7c38b3aeab', 'request_id': 'req-da1d2a68-d766-477f-84f3-0a436b0ce987'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:29.906108 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5988bcb057b4797a7f7df7c38b3aeab', 'request_id': 'req-da1d2a68-d766-477f-84f3-0a436b0ce987'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:29.912221 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5988bcb057b4797a7f7df7c38b3aeab', 'request_id': 'req-da1d2a68-d766-477f-84f3-0a436b0ce987'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:30.075845 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] The token cache is being invalidate because project e5988bcb057b4797a7f7df7c38b3aeab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:59:30.076157 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:30.076782 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:59:30.078281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1427/2881] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 18:59:29 2022] DELETE /identity/v3/projects/e5988bcb057b4797a7f7df7c38b3aeab => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:59:39.264693 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=38 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:40.071889 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:44.479927 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f401cd-b1da-4089-9b76-24f272eef2b1 None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:44.480459 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f401cd-b1da-4089-9b76-24f272eef2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:44.480459 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f401cd-b1da-4089-9b76-24f272eef2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:44.500712 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-a9f401cd-b1da-4089-9b76-24f272eef2b1 None None] MFA Rules not processed for user `dbed10f27ab548809755b32d0456d1bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:44.501968 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a9f401cd-b1da-4089-9b76-24f272eef2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:44.554591 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1455/2882] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 18:59:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:59:48.760756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:48.760756 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:48.828308 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d2a68-d766-477f-84f3-0a436b0ce987 admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:48.831882 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9237f0a-1dd7-4bb7-9dd5-0cbf4018c635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:48.832484 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9237f0a-1dd7-4bb7-9dd5-0cbf4018c635 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:48.832763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9237f0a-1dd7-4bb7-9dd5-0cbf4018c635 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:48.833031 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9237f0a-1dd7-4bb7-9dd5-0cbf4018c635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:48.834765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a9237f0a-1dd7-4bb7-9dd5-0cbf4018c635 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:48.846305 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9237f0a-1dd7-4bb7-9dd5-0cbf4018c635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:48.846579 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9237f0a-1dd7-4bb7-9dd5-0cbf4018c635 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:48.854486 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1428/2883] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 18:59:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:59:54.548031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=25 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:58.850856 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=26 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:01.616250 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f401cd-b1da-4089-9b76-24f272eef2b1 None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:01.617918 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-a9f401cd-b1da-4089-9b76-24f272eef2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:01.693235 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f401cd-b1da-4089-9b76-24f272eef2b1 None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:01.698832 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cc54c-c896-4cce-9820-146e6e1e9385 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:01.699614 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cc54c-c896-4cce-9820-146e6e1e9385 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:01.699765 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cc54c-c896-4cce-9820-146e6e1e9385 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:01.699986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cc54c-c896-4cce-9820-146e6e1e9385 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:01.701941 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-5f4cc54c-c896-4cce-9820-146e6e1e9385 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:01.714999 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4cc54c-c896-4cce-9820-146e6e1e9385 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:01.716623 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4cc54c-c896-4cce-9820-146e6e1e9385 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:01.727742 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1456/2884] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:00:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:00:04.500723 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9237f0a-1dd7-4bb7-9dd5-0cbf4018c635 service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:04.502281 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-a9237f0a-1dd7-4bb7-9dd5-0cbf4018c635 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:04.512725 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9237f0a-1dd7-4bb7-9dd5-0cbf4018c635 service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:04.516923 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17602e98-6788-4da0-bd1f-7da17256e8f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:04.517512 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17602e98-6788-4da0-bd1f-7da17256e8f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:04.517824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17602e98-6788-4da0-bd1f-7da17256e8f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:04.518100 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17602e98-6788-4da0-bd1f-7da17256e8f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:04.519780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-17602e98-6788-4da0-bd1f-7da17256e8f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:04.581929 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17602e98-6788-4da0-bd1f-7da17256e8f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:04.583365 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17602e98-6788-4da0-bd1f-7da17256e8f6 service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:04.592967 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1429/2885] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:00:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:00:11.723697 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=34 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:14.588662 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=30 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:07.249433 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cc54c-c896-4cce-9820-146e6e1e9385 service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:07.262561 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cc54c-c896-4cce-9820-146e6e1e9385 service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:07.268705 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef354f0-678f-4fb0-b87b-4f4cb1f211d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:07.278981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef354f0-678f-4fb0-b87b-4f4cb1f211d0 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:07.278981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef354f0-678f-4fb0-b87b-4f4cb1f211d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:07.278981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef354f0-678f-4fb0-b87b-4f4cb1f211d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:07.278981 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-eef354f0-678f-4fb0-b87b-4f4cb1f211d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:07.286592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef354f0-678f-4fb0-b87b-4f4cb1f211d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:07.288202 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef354f0-678f-4fb0-b87b-4f4cb1f211d0 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:07.299323 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1457/2886] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:02:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:02:09.431883 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56833a06-7403-4da2-8ecb-984b0749f7b2 None None] REQUEST_METHOD: `POST` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:09.432421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56833a06-7403-4da2-8ecb-984b0749f7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:09.432421 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56833a06-7403-4da2-8ecb-984b0749f7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:09.451812 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.auth.core [None req-56833a06-7403-4da2-8ecb-984b0749f7b2 None None] MFA Rules not processed for user `dbed10f27ab548809755b32d0456d1bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:09.453001 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-56833a06-7403-4da2-8ecb-984b0749f7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:09.460137 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1430/2887] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 19:02:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:02:09.774060 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef354f0-678f-4fb0-b87b-4f4cb1f211d0 service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:09.783355 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef354f0-678f-4fb0-b87b-4f4cb1f211d0 service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:09.787390 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2babb-4bfe-4a75-a0f0-8b604bfbb10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:09.787979 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2babb-4bfe-4a75-a0f0-8b604bfbb10b service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:09.788263 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2babb-4bfe-4a75-a0f0-8b604bfbb10b service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:09.788535 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2babb-4bfe-4a75-a0f0-8b604bfbb10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:09.790310 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cff2babb-4bfe-4a75-a0f0-8b604bfbb10b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:09.799616 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2babb-4bfe-4a75-a0f0-8b604bfbb10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:09.801031 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2babb-4bfe-4a75-a0f0-8b604bfbb10b service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:09.808976 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1458/2888] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:02:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:02:10.523635 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56833a06-7403-4da2-8ecb-984b0749f7b2 None None] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:10.525042 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-56833a06-7403-4da2-8ecb-984b0749f7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:10.598656 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56833a06-7403-4da2-8ecb-984b0749f7b2 None None] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:10.603669 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20673278-9e4e-4d93-9040-23ab61a55d48 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0aa6b215f646c280af245a89ccf849', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:10.604269 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20673278-9e4e-4d93-9040-23ab61a55d48 service placement] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:10.604551 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20673278-9e4e-4d93-9040-23ab61a55d48 service placement] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:10.604824 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20673278-9e4e-4d93-9040-23ab61a55d48 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:10.606570 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-20673278-9e4e-4d93-9040-23ab61a55d48 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:10.617780 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20673278-9e4e-4d93-9040-23ab61a55d48 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:10.619279 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20673278-9e4e-4d93-9040-23ab61a55d48 service placement] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:10.628968 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1431/2889] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:02:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:02:19.805412 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=12 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:20.624830 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=35 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:31.464925 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2babb-4bfe-4a75-a0f0-8b604bfbb10b service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:31.466393 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-cff2babb-4bfe-4a75-a0f0-8b604bfbb10b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:31.482728 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2babb-4bfe-4a75-a0f0-8b604bfbb10b service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:31.488233 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ab7f5-d1bd-46da-a8df-b840affeb486 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:31.488828 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ab7f5-d1bd-46da-a8df-b840affeb486 service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:31.489107 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ab7f5-d1bd-46da-a8df-b840affeb486 service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:31.489378 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ab7f5-d1bd-46da-a8df-b840affeb486 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:31.491121 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-038ab7f5-d1bd-46da-a8df-b840affeb486 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:31.504983 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ab7f5-d1bd-46da-a8df-b840affeb486 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:31.506634 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ab7f5-d1bd-46da-a8df-b840affeb486 service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:31.517808 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1459/2890] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:02:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:02:41.513175 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:58.262539 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20673278-9e4e-4d93-9040-23ab61a55d48 service placement] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:58.273366 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20673278-9e4e-4d93-9040-23ab61a55d48 service placement] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:58.279062 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c5342-4558-4a8e-a5be-aa004d77fa0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0aa6b215f646c280af245a89ccf849', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:58.279697 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c5342-4558-4a8e-a5be-aa004d77fa0d service placement] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:58.279981 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c5342-4558-4a8e-a5be-aa004d77fa0d service placement] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:58.280257 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c5342-4558-4a8e-a5be-aa004d77fa0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:58.282018 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f51c5342-4558-4a8e-a5be-aa004d77fa0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:58.294301 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c5342-4558-4a8e-a5be-aa004d77fa0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:58.295873 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c5342-4558-4a8e-a5be-aa004d77fa0d service placement] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:58.305815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1432/2891] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:02:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:03:08.301541 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=6 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:08.327880 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c5d89-e1f6-4221-92a4-99b83b616e6d None None] REQUEST_METHOD: `POST` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:08.328448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c5d89-e1f6-4221-92a4-99b83b616e6d None None] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:08.328448 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c5d89-e1f6-4221-92a4-99b83b616e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:08.347695 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.auth.core [None req-179c5d89-e1f6-4221-92a4-99b83b616e6d None None] MFA Rules not processed for user `dbed10f27ab548809755b32d0456d1bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:04:08.348904 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-179c5d89-e1f6-4221-92a4-99b83b616e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:08.355768 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1460/2892] 10.222.0.57 () {58 vars in 958 bytes} [Tue Aug 9 19:04:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:04:08.394456 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c5342-4558-4a8e-a5be-aa004d77fa0d service placement] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:08.395887 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-f51c5342-4558-4a8e-a5be-aa004d77fa0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:08.454346 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c5342-4558-4a8e-a5be-aa004d77fa0d service placement] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:08.457922 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05df4b-3bc8-40cc-8870-8f4462873531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77300e25b9004a598f8f7f12f3cf4866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:08.458525 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05df4b-3bc8-40cc-8870-8f4462873531 service cinder] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:08.458815 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05df4b-3bc8-40cc-8870-8f4462873531 service cinder] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:08.459091 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05df4b-3bc8-40cc-8870-8f4462873531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:08.460841 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-5f05df4b-3bc8-40cc-8870-8f4462873531 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:08.474939 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05df4b-3bc8-40cc-8870-8f4462873531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:08.476363 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05df4b-3bc8-40cc-8870-8f4462873531 service cinder] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:08.487487 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1433/2893] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:04:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:04:08.777413 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c5d89-e1f6-4221-92a4-99b83b616e6d None None] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:08.787664 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c5d89-e1f6-4221-92a4-99b83b616e6d None None] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:08.791163 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60b14b-e439-41dc-ae59-b794078e86b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:08.791758 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60b14b-e439-41dc-ae59-b794078e86b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:08.792035 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60b14b-e439-41dc-ae59-b794078e86b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:08.792301 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60b14b-e439-41dc-ae59-b794078e86b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:08.794014 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-ea60b14b-e439-41dc-ae59-b794078e86b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:08.803353 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60b14b-e439-41dc-ae59-b794078e86b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:08.804781 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60b14b-e439-41dc-ae59-b794078e86b5 service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:08.813345 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1461/2894] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:04:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:04:09.469797 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05df4b-3bc8-40cc-8870-8f4462873531 service cinder] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:09.479064 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05df4b-3bc8-40cc-8870-8f4462873531 service cinder] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:09.482565 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bba27-f3c2-4ae9-8c65-36c479f8602b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:09.483179 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bba27-f3c2-4ae9-8c65-36c479f8602b service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:09.483431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bba27-f3c2-4ae9-8c65-36c479f8602b service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:09.483699 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bba27-f3c2-4ae9-8c65-36c479f8602b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:09.486057 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-1f5bba27-f3c2-4ae9-8c65-36c479f8602b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:09.501431 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bba27-f3c2-4ae9-8c65-36c479f8602b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:09.503539 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bba27-f3c2-4ae9-8c65-36c479f8602b service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:09.514488 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1434/2895] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:04:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:04:18.809182 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=9 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:19.509538 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=32 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:25.732694 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60b14b-e439-41dc-ae59-b794078e86b5 service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:25.746481 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60b14b-e439-41dc-ae59-b794078e86b5 service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:25.752704 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995df04f-f426-44ea-9926-85ace606d42f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ecd7d1cce644b0982cc23740537933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:25.753298 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995df04f-f426-44ea-9926-85ace606d42f service nova] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:25.753607 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995df04f-f426-44ea-9926-85ace606d42f service nova] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:25.753880 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995df04f-f426-44ea-9926-85ace606d42f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:25.755608 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-995df04f-f426-44ea-9926-85ace606d42f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:25.765976 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995df04f-f426-44ea-9926-85ace606d42f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:25.767553 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995df04f-f426-44ea-9926-85ace606d42f service nova] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:25.776306 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1462/2896] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:04:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:04:35.772326 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=6 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:04.020369 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bba27-f3c2-4ae9-8c65-36c479f8602b service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:05:04.029072 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bba27-f3c2-4ae9-8c65-36c479f8602b service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:05:04.032609 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa43565-ea4a-459a-940a-edac2ebbeb2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:05:04.033193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa43565-ea4a-459a-940a-edac2ebbeb2d service neutron] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:04.033466 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa43565-ea4a-459a-940a-edac2ebbeb2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:04.033763 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa43565-ea4a-459a-940a-edac2ebbeb2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:04.040894 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa43565-ea4a-459a-940a-edac2ebbeb2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:05:04.042342 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa43565-ea4a-459a-940a-edac2ebbeb2d service neutron] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:05:04.050710 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1435/2897] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:05:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:05:14.046886 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=6 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:11.055775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995df04f-f426-44ea-9926-85ace606d42f service nova] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:11.066046 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995df04f-f426-44ea-9926-85ace606d42f service nova] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:11.069905 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ec33f-0e8f-45d9-b528-e008fb82296e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a7cdd610f74330a5da69c0de8472db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:11.070499 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ec33f-0e8f-45d9-b528-e008fb82296e service neutron] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:11.070775 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ec33f-0e8f-45d9-b528-e008fb82296e service neutron] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:11.071041 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ec33f-0e8f-45d9-b528-e008fb82296e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:11.072717 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e29ec33f-0e8f-45d9-b528-e008fb82296e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:11.135851 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ec33f-0e8f-45d9-b528-e008fb82296e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:11.137287 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ec33f-0e8f-45d9-b528-e008fb82296e service neutron] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:11.147314 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1463/2898] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:14.782724 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa43565-ea4a-459a-940a-edac2ebbeb2d service neutron] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:14.795258 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa43565-ea4a-459a-940a-edac2ebbeb2d service neutron] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:14.800492 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf23e5c-fdee-40a0-a84f-9ae1f18a1d72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0aa6b215f646c280af245a89ccf849', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:14.801138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf23e5c-fdee-40a0-a84f-9ae1f18a1d72 service placement] REQUEST_METHOD: `GET` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:14.801496 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf23e5c-fdee-40a0-a84f-9ae1f18a1d72 service placement] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:14.801811 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf23e5c-fdee-40a0-a84f-9ae1f18a1d72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:14.803779 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-ccf23e5c-fdee-40a0-a84f-9ae1f18a1d72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:14.815765 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf23e5c-fdee-40a0-a84f-9ae1f18a1d72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:14.817404 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf23e5c-fdee-40a0-a84f-9ae1f18a1d72 service placement] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:14.828284 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1436/2899] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:18.279516 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ec33f-0e8f-45d9-b528-e008fb82296e service neutron] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:18.280940 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-e29ec33f-0e8f-45d9-b528-e008fb82296e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:18.293675 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ec33f-0e8f-45d9-b528-e008fb82296e service neutron] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:18.298917 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:18.299593 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:18.299816 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:18.300085 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] PATH_INFO: `/v3/users/dbed10f27ab548809755b32d0456d1bc` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:18.301193 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbed10f27ab548809755b32d0456d1bc)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:18.303166 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:18.433685 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbed10f27ab548809755b32d0456d1bc'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:18.510236 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbed10f27ab548809755b32d0456d1bc'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:18.510894 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbed10f27ab548809755b32d0456d1bc'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:18.518703 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbed10f27ab548809755b32d0456d1bc'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:18.530545 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbed10f27ab548809755b32d0456d1bc'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:18.537212 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1464/2900] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 19:06:18 2022] DELETE /identity/v3/users/dbed10f27ab548809755b32d0456d1bc => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:06:18.769347 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf23e5c-fdee-40a0-a84f-9ae1f18a1d72 service placement] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:18.780336 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf23e5c-fdee-40a0-a84f-9ae1f18a1d72 service placement] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:18.839231 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:18.839769 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:18.840045 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:18.840313 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] PATH_INFO: `/v3/projects/098c57b11d7948c8a1ef732af98ec75f` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:18.841447 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] RBAC: Authorizing `identity:delete_project(project_id=098c57b11d7948c8a1ef732af98ec75f)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:18.843393 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:18.939755 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '098c57b11d7948c8a1ef732af98ec75f', 'request_id': 'req-03a6620c-7748-41f1-b660-6d65e45ae30c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:18.946132 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '098c57b11d7948c8a1ef732af98ec75f', 'request_id': 'req-03a6620c-7748-41f1-b660-6d65e45ae30c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:18.951587 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '098c57b11d7948c8a1ef732af98ec75f', 'request_id': 'req-03a6620c-7748-41f1-b660-6d65e45ae30c'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:19.023661 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] The token cache is being invalidate because project 098c57b11d7948c8a1ef732af98ec75f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:06:19.023951 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:19.024577 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:19.026079 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1437/2901] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 19:06:18 2022] DELETE /identity/v3/projects/098c57b11d7948c8a1ef732af98ec75f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:06:19.030089 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:19.031527 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:19.044582 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0748135-5dd1-489f-8723-98ad276781c0 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:19.101313 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:19.101902 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:19.102186 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:19.102464 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] PATH_INFO: `/v3/users/dc87fff1029748a784868170411309ca` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:19.103602 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc87fff1029748a784868170411309ca)` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:19.105539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:19.110986 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:19.302648 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc87fff1029748a784868170411309ca'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:19.373224 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc87fff1029748a784868170411309ca'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:19.373787 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc87fff1029748a784868170411309ca'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:19.382042 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc87fff1029748a784868170411309ca'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:19.393726 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.notifications [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc87fff1029748a784868170411309ca'} {{(pid=80446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:19.399539 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1465/2902] 10.222.0.57 () {58 vars in 1224 bytes} [Tue Aug 9 19:06:19 2022] DELETE /identity/v3/users/dc87fff1029748a784868170411309ca => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:06:19.911394 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] Authenticating user token {{(pid=80445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:19.937903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.fernet_utils [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:19.937903 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6620c-7748-41f1-b660-6d65e45ae30c admin admin] Validating token access rules against request {{(pid=80445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:19.985227 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a379f0e1f1f4c9c884081f682310ad9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee70c279239f4e719fad35784db1a619', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:19.985879 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:19.986159 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:19.986429 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d admin admin] PATH_INFO: `/v3/projects/42899db38b484d4283e223a722029c3a` {{(pid=80445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:19.987530 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=42899db38b484d4283e223a722029c3a)` {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:19.989401 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d admin admin] RBAC: Authorization granted {{(pid=80445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:20.010138 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:20.076442 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42899db38b484d4283e223a722029c3a', 'request_id': 'req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:20.083019 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42899db38b484d4283e223a722029c3a', 'request_id': 'req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:20.088448 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42899db38b484d4283e223a722029c3a', 'request_id': 'req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d'} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:20.189239 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d admin admin] The token cache is being invalidate because project 42899db38b484d4283e223a722029c3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:06:20.189470 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:20.190084 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG keystone.notifications [None req-ff805527-af52-44e6-bb9a-ba9f5a88cb8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:06:20.191552 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: [pid: 80445|app: 0|req: 1438/2903] 10.222.0.57 () {58 vars in 1233 bytes} [Tue Aug 9 19:06:19 2022] DELETE /identity/v3/projects/42899db38b484d4283e223a722029c3a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:06:29.396135 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:30.186193 dv-cinder-826608-9-iscsi devstack@keystone.service[80445]: DEBUG dbcounter [-] [80445] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:06.245087 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] Authenticating user token {{(pid=80446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:08:06.246592 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:06.388556 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446db2f-8bab-4e74-9255-3c7a8e745d39 admin admin] Validating token access rules against request {{(pid=80446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:08:06.447821 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954eef8-136d-4088-b0de-fc24a63482a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0aa6b215f646c280af245a89ccf849', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abcccf694e7b4f66a7372e0ede416caf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:08:06.448423 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954eef8-136d-4088-b0de-fc24a63482a9 service placement] REQUEST_METHOD: `GET` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:06.448706 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954eef8-136d-4088-b0de-fc24a63482a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:06.448980 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954eef8-136d-4088-b0de-fc24a63482a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:06.450750 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.fernet_utils [None req-f954eef8-136d-4088-b0de-fc24a63482a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:06.462604 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954eef8-136d-4088-b0de-fc24a63482a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:08:06.464357 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954eef8-136d-4088-b0de-fc24a63482a9 service placement] RBAC: Authorization granted {{(pid=80446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:08:06.474962 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: [pid: 80446|app: 0|req: 1466/2904] 10.222.0.57 () {60 vars in 1365 bytes} [Tue Aug 9 19:08:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:08:16.470267 dv-cinder-826608-9-iscsi devstack@keystone.service[80446]: DEBUG dbcounter [-] [80446] Writing DB stats keystone:SELECT=89 {{(pid=80446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}